Efficient Training and Accuracy Improvement of Imaging Based Assay

ABSTRACT

The present disclosure relates to devices, apparatus and methods of improving the accuracy of image-based assay, that uses imaging system having uncertainties or deviations (imperfection) compared with an ideal imaging system. One aspect of the present invention is to add the monitoring marks on the sample holder, with at least one of their geometric and/optical properties of the monitoring marks under predetermined and known, and taking images of the sample with the monitoring marks, and train a machine learning model using the images with the monitoring mark.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/431,345 filed Aug. 16, 2021, which is the U.S. national stage ofPCT/US2020/062445 filed Nov. 25, 2020, which claims priority benefit toU.S. Provisional Application 62/940,242 filed Nov. 25, 2019. Thecontents of the above-mentioned applications are incorporated herein byreference in their entireties.

FIELD

The present disclosure relates to devices, apparatus and methods ofimproving the accuracy of image-based assay, that uses imaging systemhaving uncertainties or deviations (imperfection) compared with an idealimaging system.

BACKGROUND

In image-based bio/chemical sensing and assaying (e.g., immunoassay,nucleotide assay, blood cell counting, etc.), for low cost and/orportable system, a low quality imaging system often used. However, a lowquality imaging system can have an imperfection (an deviation from anideal) in optical elements, mechanical system, and/or electrical system.Such imperfection can significantly affect the assay accuracy.

To improve an assay accuracy in an image-based assay using a low qualityimaging system, machine learning methods can be used. However, simplyjust machine learning alone, the imperfection in imaging system createsmany variable, requiring long training time, large training samples, andartifacts in imaging (hence new errors). One aspect of the presentinvention is to add the monitoring marks on the sample holder, with atleast one of their geometric and/optical properties of the monitoringmarks under predetermined and known, and taking images of the samplewith the monitoring marks, and train a machine learning model using theimages with the monitoring mark.

SUMMARY

One aspect of the present invention is to add the monitoring marks onthe sample holder, with at least one of their geometric and/opticalproperties of the monitoring marks under predetermined and known, andtaking images of the sample with the monitoring marks, and train amachine learning model using the images with the monitoring mark.

In some embodiments, the present invention provides a method of traininga machine learning model for an image based assay, wherein the assay is,during a test, imaged by a low-quality imaging system, the methodincluding the steps of: having a first sample forming a thin layer on animaging area of a first sample holder, wherein the first sample holderis a marked sample holder comprising one or more monitoring marks on theimaging area; having a second sample forming a thin layer on an imagingarea of a second sample holder, wherein the second sample holder is amarked sample holder that comprising an identical one or more monitoringmarks on the imaging area of the second sample holder to the one or moremonitoring marks on the first sample holder; imaging, using alow-quality imaging system, a first image of the sample on the imagingarea of the first sample holder; imaging; using a high-quality imagingsystem, a second image of the sample on the imaging area of the secondsample holder; correcting an imperfection in the first image using themonitoring marks, generating a first corrected image; correcting animperfection in the second image using the monitoring marks, if thesecond image has an imperfection, generating a second corrected image;and training a machine learning model using the first corrected image,the second corrected image and the monitoring marks, generating atrained model, wherein a geometric property and optionally an opticalproperty of the one or more monitoring marks imaged under an idealimaging system is predetermined and known; wherein a low quality imagingsystem comprise more imperfection than a high quality imaging system.

In some embodiments, the method includes having a third sample forming athin layer on an imaging area of a third sample holder, wherein thethird sample holder is a marked sample holder that comprising one ormore monitoring marks on the imaging area of the third sample holderidentical to the one or more monitoring marks on the first sampleholder; imaging, using a low-quality imaging system, a third image ofthe samples on the imaging area of the third sample holder; correcting,using the monitoring marks, an imperfection in the third image,generating a corrected third image; and analyzing the transformedcorrected third image using the machine learning model trained in claim1 and generating an assay result.

In some embodiments, the machine learning model comprises a cyclegenerative adversarial network (CycleGAN).

In some embodiments, the machine learning model comprises a cyclegenerative adversarial network (CycleGAN) comprising a forwardgenerative adversarial network (forward GAN) and a backward GAN, whereinthe forward GAN comprises a first generator and a first discriminator,and the backward GAN comprises a second generator and a seconddiscriminator, and wherein training the machine learning model usingeach of transformed regions in the first image and each of transformedregions in the second image comprises training the CycleGAN using eachof transformed regions in the first image and each of transformedregions in the second image registered at four structural elements atfour corners of the corresponding regions.

In some embodiments, the first sample and the second sample are the samesample, and the first sample holder and the second sample holder are thesame.

In some embodiments, the present invention provides a method to train amachine learning model for image-based assays, the method including thesteps of: receiving a first image, captured by a first optical sensor,of a sample holder containing a sample, wherein the sample holder isfabricated with a standard of patterned structural elements atpredetermined positions; identifying a first region in the first imagebased on locations of one or more structural elements of the patternedstructural elements in the first image; determining a spatial transformassociated with the first region based on a mapping between thelocations of the one or more structural elements in the first image andpredetermined positions of one or more structural elements in the sampleholder; applying the spatial transform to the first region in the firstimage to calculate a transformed first region; and training the machinelearning model using the transformed first image.

In some embodiments, the sample holder comprises a first plate, a secondplate, and the patterned structural elements, and wherein the patternedstructural elements comprise pillars embedded at the predeterminedpositions on at least one of the first plate or the second plate.

In some embodiments, the method further includes the steps of: detectingthe locations of the patterned structural elements in the first image;partitioning the first image into regions comprising the first region,wherein each of the regions is defined by four structural elements atfour corners of the corresponding region; determining a correspondingspatial transform associated with each of the regions in the first imagebased on a mapping between the locations of the four structural elementsat the four corners of the corresponding region and the fourpredetermined positions of the four structural elements in the sampleholder; applying the corresponding spatial transform to each of theregions in the first image to calculate a corresponding transformedregion in the first image; and training the machine learning model usingeach of transformed regions in the first image, wherein the trainedmachine learning model is used to transform assay images from a lowresolution to a high resolution.

In some embodiments, the predetermined positions of the patternedstructural elements are distributed periodically with at least oneperiodicity value, and wherein detecting the locations of the patternedstructural elements in the first image includes: detecting, using asecond machine learning model, the locations of the patterned structuralelements in the first image; and correcting, based on the at least oneperiodicity value, an error in the detected locations of the patternedstructural elements in the first image.

In some embodiments, the method further includes the steps of: receivinga second image of the sample holder captured by a second optical sensor,wherein the first image is captured at a first quality level and thesecond image is captured at a second quality level which is higher thanthe first quality level; partitioning the second image into regions,wherein each of the regions in the second image is defined by fourstructural elements at four corners of the corresponding region in thesecond image and is matched to a corresponding region in the firstimage; determining a second spatial transform associated with a regionin the second image based on a mapping between the locations of the fourstructural elements at the four corners of the corresponding region inthe second image and the four predetermined positions of the fourstructural elements in the sample holder; applying the second spatialtransform to each of the regions in the second image to calculate acorresponding transformed region in the second image; and training themachine learning model from transforming first quality level images tosecond quality level images using each of transformed regions in thefirst image and each of transformed regions in the second image.

In some embodiments, the machine learning model comprises a cyclegenerative adversarial network (CycleGAN) comprising a forwardgenerative adversarial network (forward GAN) and a backward GAN, whereinthe forward GAN comprises a first generator and a first discriminator,and the backward GAN comprises a second generator and a seconddiscriminator, and wherein training the machine learning model usingeach of transformed regions in the first image and each of transformedregions in the second image comprises training the CycleGAN using eachof transformed regions in the first image and each of transformedregions in the second image registered at four structural elements atfour corners of the corresponding regions.

In some embodiments, training the machine learning model using each oftransformed regions in the first image and each of transformed regionsin the second image includes: training the first generator and the firstdiscriminator by providing each of transformed regions in the firstimage to the forward GAN; training the second generator and the seconddiscriminator by providing each of transformed regions in the secondimage to the backward GAN; and optimizing the forward and backward GANtraining under a cycle consistency constraint.

In some embodiments, the present invention provides a method forconverting an assay image using a machine learning model, the methodincluding the steps of: receiving a first image, captured by a firstoptical sensor, of a sample holder containing a sample, wherein thesample holder is fabricated with a standard of patterned structuralelements at predetermined positions; identifying a first region in thefirst image based on locations of one or more structural elements of thepatterned structural elements in the first image; determining a spatialtransform associated with the first region based on a mapping betweenthe locations of the one or more structural elements in the first imageand predetermined positions of one or more structural elements in thesample holder; applying the spatial transform to the first region in thefirst image to calculate a transformed first region; and applying themachine learning model to the transformed first region in the firstimage to generate a second region.

In some embodiments, the method further includes the steps of:partitioning the first image into a plurality of regions based thelocations of the one or more structural elements of the patternedstructural elements in the first image, wherein the plurality of regionscomprises the first region; determining a respective spatial transformassociated with each of the plurality of regions; applying thecorresponding spatial transform to each of the plurality of regions inthe first image to calculate transformed regions; applying the machinelearning model to each of the transformed regions in the first image togenerate transformed regions of a second quality level; and combiningthe transformed regions to form a second image.

In some embodiments, the present invention provides an image-based assaysystem including: a database system to store images; and a processingdevice, communicatively coupled to the database system, to: receive afirst image, captured by a first optical sensor, of a sample holdercontaining a sample, wherein the sample holder is fabricated with astandard of patterned structural elements at predetermined positions;identify a first region in the first image based on locations of one ormore structural elements of the patterned structural elements in thefirst image; determine a spatial transform associated with the firstregion based on a mapping between the locations of the one or morestructural elements in the first image and predetermined positions ofone or more structural elements in the sample holder; apply the spatialtransform to the first region in the first image to calculate atransformed first region; and train the machine learning model using thetransformed first image.

In some embodiments, the sample holder comprises a first plate, a secondplate, and the patterned structural elements, and wherein the patternedstructural elements comprise pillars embedded at the predeterminedpositions on at least one of the first plate or the second plate.

In some embodiments, the processing device is further to: detect thelocations of the patterned structural elements in the first image;partition the first image into regions comprising the first region,wherein each of the regions is defined by four structural elements atfour corners of the corresponding region; determine a correspondingspatial transform associated with each of the regions in the first imagebased on a mapping between the locations of the four structural elementsat the four corners of the corresponding region and the fourpredetermined positions of the four structural elements in the sampleholder; apply the corresponding spatial transform to each of the regionsin the first image to calculate a corresponding transformed region inthe first image; and train the machine learning model using each oftransformed regions in the first image, wherein the trained machinelearning model is used to transform assay images from a low resolutionto a high resolution.

In some embodiments, the predetermined positions of the patternedstructural elements are distributed periodically with at least oneperiodicity value, and wherein to detect the locations of the patternedstructural elements in the first image, the processing device is furtherto: detect, using a second machine learning model, the locations of thepatterned structural elements in the first image; and correct, based onthe at least one periodicity value, an error in the detected locationsof the patterned structural elements in the first image.

In some embodiments, the processing device is further to: receive asecond image of the sample holder captured by a second optical sensor,wherein the first image is captured at a first quality level and thesecond image is captured at a second quality level which is higher thanthe first quality level; partition the second image into regions,wherein each of the regions in the second image is defined by fourstructural elements at four corners of the corresponding region in thesecond image and is matched to a corresponding region in the firstimage; determine a second spatial transform associated with a region inthe second image based on a mapping between the locations of the fourstructural elements at the four corners of the corresponding region inthe second image and the four predetermined positions of the fourstructural elements in the sample holder; apply the second spatialtransform to each of the regions in the second image to calculate acorresponding transformed region in the second image; and train themachine learning model from transforming first quality level images tosecond quality level images using each of transformed regions in thefirst image and each of transformed regions in the second image.

In some embodiments, the machine learning model comprises a cyclegenerative adversarial network (CycleGAN) comprising a forwardgenerative adversarial network (forward GAN) and a backward GAN, whereinthe forward GAN comprises a first generator and a first discriminator,and the backward GAN comprises a second generator and a seconddiscriminator, and wherein training the machine learning model usingeach of transformed regions in the first image and each of transformedregions in the second image comprises training the CycleGAN using eachof transformed regions in the first image and each of transformedregions in the second image registered at four structural elements atfour corners of the corresponding regions.

In some embodiments, to train the machine learning model using each oftransformed regions in the first image and each of transformed regionsin the second image, the processing device is further to: train thefirst generator and the first discriminator by providing each oftransformed regions in the first image to the forward GAN; and train thesecond generator and the second discriminator by providing each oftransformed regions in the second image to the backward GAN.

In some embodiments, the present invention provides an image-based assaysystem for converting an assay image using a machine learning model,including: a database system to store images; and a processing device,communicatively coupled to the database system, to: receive a firstimage, captured by a first optical sensor, of a sample holder containinga sample, wherein the sample holder is fabricated with a standard ofpatterned structural elements at predetermined positions; identify afirst region in the first image based on locations of one or morestructural elements of the patterned structural elements in the firstimage; determine a spatial transform associated with the first regionbased on a mapping between the locations of the one or more structuralelements in the first image and predetermined positions of one or morestructural elements in the sample holder; apply the spatial transform tothe first region in the first image to calculate a transformed firstregion; and apply the machine learning model to the transformed firstregion in the first image to generate a second region.

In some embodiments, the processing device is further to: partition thefirst image into a plurality of regions based the locations of the oneor more structural elements of the patterned structural elements in thefirst image, wherein the plurality of regions comprises the firstregion; determine a respective spatial transform associated with each ofthe plurality of regions; apply the corresponding spatial transform toeach of the plurality of regions in the first image to calculatetransformed regions; apply the machine learning model to each of thetransformed regions in the first image to generate transformed regionsof a second quality level; and combine the transformed regions to form asecond image.

In some embodiment, the sample holder has one plate, and the sample iscontacting one surface of the plate.

In some embodiment, the sample holder has two plate, and the sample isbetween the two plates.

In some embodiment, the sample holder has two plate that is movable toeach other, and the sample is between the two plates.

In some embodiment, the sample holder has two plate that is movable toeach other, and the sample is between the two plates. A plurality ofspacers attached to at least one interior opposing surface of at leastone of the plates, or both, and the plurality of spacers are situatedbetween the opposable plates. The sample thickness is regulated by thespacers.

In some embodiments, there is at least one spacer inside the sample.

In some embodiment, the spacers are the monitoring markers.

In certain embodiments, the two plates of the device are initially ontop of each other and need to be separated to get into an openconfiguration for sample deposition.

In certain embodiments, the two plates of the device are already in theclosed configuration before the sample deposition. The sample gets intothe device from a gap between the two plates.

In some embodiment, the thickness of the sample layer is 0.1 um, 0.5 um,1 um, 2 um, 3 um, 5 um, 10 um, 50 um, 100 um, 200 um, 500 um, 1000 um,5000 um or in a range between any of the two values.

In some embodiment, the preferred thickness of the sample layer is 1 um,2 um, 5 um, 10 um, 30 um, 50 um, 100 um, 200 um, or in a range betweenany of the two values.

In certain embodiments, the spacing between two monitoring markers is 1um, 2 um, 3 um, 5 um, 10 um, 50 um, 100 um, 200 um, 500 um, 1000 um,5000 um or in a range between any of the two values.

In certain embodiments, the preferred spacing between two monitoringmarkers is 10 um, 50 um, 100 um, 200 um, or in a range between any ofthe two values.

In certain embodiments, the spacing between two monitoring markers is10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% or in a range betweenany of the two values of the lateral dimension of imaging area.

In certain embodiments, the preferred spacing between two monitoringmarkers is 30%, 50%, 80%, 90% or in a range between any of the twovalues of the lateral dimension of imaging area.

In certain embodiments, the average size monitoring markers is 1 um, 2um, 3 um, 5 um, 10 um, 50 um, 100 um, 200 um, 500 um, 1000 um, or in arange between any of the two values.

In certain embodiments, the preferred average size monitoring markers is5 um, 10 um, 50 um, 100 um, or in a range between any of the two values.

In certain embodiments, the average size monitoring markers is 1%, 10%,20%, 30%, 40%, 50%, 60%, 70%, 80%, or in a range between any of the twovalues of the size of imaging area.

In certain embodiments, the preferred average size monitoring markers is1%, 10%, 20%, 30%, or in a range between any of the two values of thesize of imaging area.

In certain embodiment, the spacers are the monitoring markers with aheight of 0.1 um, 0.5 um, 1 um, 2 um, 3 um, 5 um, 10 um, 50 um, 100 um,200 um, 500 um, 1000 um, 5000 um or in a range between any of the twovalues.

In certain embodiment, the spacers are the monitoring markers with apreferred height of 1 um, 2 um, 5 um, 10 um, 30 um, 50 um, 100 um, 200um or in a range between any of the two values.

In certain embodiment, the spacers are the monitoring markers with aheight of 1%, 5%, 10%, 20%, 40%, 60%, 80%, 100% or in a range betweenany of the two values of the height of sample layer.

In certain embodiment, the spacers are the monitoring markers with apreferred height of 50%, 60%, 80%, 100% or in a range between any of thetwo values of the height of sample layer.

In certain embodiment, the shape of the monitoring markers is selectedfrom round, polygonal, circular, square, rectangular, oval, elliptical,or any combination of the same.

In certain embodiment, the monitoring markers have pillar shape, andhave a substantially flat top surface covering at least 10% of the topprojection area of markers.

In certain embodiment, the inter monitoring markers distance isperiodic. Traditional image-based assays may employ a high-precisionmicroscope equipped with an imaging system to capture the images of theassays. These types of optical microscopes may be subject to theobjective lens scaling rule according to which the field-of-view (FoV)is inversely proportional to the magnification power. According to therule, the image for assaying with higher magnification power has asmaller FoV, and the one with larger FoV has less magnification power.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the disclosure. The drawings, however, should not betaken to limit the disclosure to the specific embodiments, but are forexplanation and understanding only.

FIG. 1 training phase: place samples on the same type of the MarkedSample Holder (MSH) and use both low-quality (LQ) Imaging System andhigh-quality (HQ) Imaging System to capture LQ and HQ image of thesample in MMH. Images in both domains are corrected using the MonitorMarkers (MM) to finalize the training data. A machine learning model istrained based on the training data for the image transformation model G.

FIG. 2 prepare the Monitor Marker Corrected Low Quality Image (MMC-LQI)follow the same procedure as training stage route A. Then feed theMMC-LQI to model G for the final result.

FIG. 3 illustrates detail description for Block 5 in training phase inFIG. 2 .

FIG. 4 illustrates an image-based assay system that may capture assayimages and process the images for training a machine learning modelaccording to an embodiment of the disclosure.

FIG. 5 depicts a flow diagram of a method to prepare the high-qualityassay images in the training dataset according to an embodiment of thedisclosure.

FIG. 6 illustrates the construction of a cycle generative adversarialnetwork (CycleGAN) model used for assay image-to-image translationaccording to an embodiment of the disclosure.

FIG. 7 depicts a flow diagram of a method to enhance a low-quality assayimage according to an embodiment of the disclosure.

FIG. 8 depicts a flow diagram of a method for preparing a trainingdataset for a machine learning model according to an embodiment of thedisclosure.

FIG. 9 depicts a block diagram of a computer system operating inaccordance with one or more aspects of the present disclosure.

FIG. 10 (A) illustrates that the monitoring marks in the presentinvention are actually adding known meta structures into a lattice. Whena lattice gets distorted, one always can use the knowledge of the metastructure to recovering to nearly perfect structure, hence greatlyimprove the accuracy of the machine learning model and the assayaccuracy.

FIG. 11 illustrate an actual examples of the difference between trainingmachine learning without using the monitoring marks (FIG. 10 (a0) whichneed many samples and long time to train and the test generatesartifacts (missing cells and create the non-exist cells), while usingthe monitoring marks, the training sample number and time aresignificantly reduced, and the test do not generate the artifacts.

DETAILED DESCRIPTION Definition

The term “a marked sample holder” or “MSH” refer to a sample holder thathas the monitoring marks.

The term “monitoring marks” or “MM” refer to one or more structures on asample holder, wherein at least one geometrical property of the one ormore structures, when viewed in an ideal imaging system is predeterminedand known, wherein the geometric property comprises a dimension and/ororientation of the one or more structure, and/or the position betweentwo of structures. In some embodiment, in additional to thepredetermined and known geometric property, an optical property of theone or more structure is predetermined and known, wherein the opticalproperty comprise light transmission, absorption, reflection, and/orscattering. optical

The term “an identical marked sample holder” refers to a second markedsample holder that is fabricated precisely to have ?? that are identicalto that of a first marked sample holder.

The term “an imaging system” refers to a system that is used to takeimage. An imaging system comprises optical elements, mechanical system,and/or electrical system, where the optical elements comprising animager (that takes an image), light illumination, lens, filter, and/ormirror, the mechanical system comprising the mechanical stage, scanner,and/or mechanical holder, and the electrical system, comprises powersupplies, wiring, and/or electronics.

The term “imperfection” refers an deviation from an ideal, wherein animperfection can be random or non-random, and/or time dependent.

The term “monitoring mark corrected” or “MC” referred to a processedimage from an original image, wherein one or more imperfection, ifexist, in the original image is corrected using

The term “correction” refers to a calculation using an algorithm.

The term “machine learning model” and “algorithm” are interchangeable.

The term “transformed image using the monitoring mark” and “correctedimage using the monitoring mark” are interchangeable.

The term “optical sensor” in the Figures and “imaging system” areexchangeable.

In certain embodiments, “a standard of patterned structural elements atpredetermined positions” are the monitoring marks.

Imperfection of Imaging system means imperfect conditions in followingelements of the system:

Optical components and conditions including but not limit to opticalattenuator, beam splitter, depolarizer, diaphragm, diffractive beamsplitter, diffuser, ground glass, lens, littrow prism, multifocaldiffractive lens, nanophotonic resonator, nuller, optical circulator,optical isolator, optical microcavity, photonic integrated circuit,pinhole (optics), polarizer, primary mirror, prism, q-plate,retroreflector, spatial filter, spatial light modulator, virtuallyimaged phased array, waveguide (optics), waveplate, zone plate;

Light illumination and conditions including but not limit to lightsource intensity, light source spectra, light source color, light sourcedirection, light source brightness, light source contrast, light sourcewavelength band width, light source coherence, light source phase, lightsource polarization;

Image sensor and photo detector components and conditions including butnot limit to CCD (charge coupled device) and CMOS (complementary metaloxide semiconductor) image sensors' exposure-time, color separation,resolution, ISO, noise level, and sensitivity;

Mechanic components and conditions including but not limit to sampleholder (flatness, parallelism, surface roughness, distance to lens),scanning system (flatness, parallelism, stiffness, resolution, travel),material stability, material thermal expansion;

Image from Imaging system conditions including but not limit tospherical distortions, noise level, resolution, brightness distribution,contrast distribution, color distribution, temperature distribution, huedistribution, saturation, lightness, rotation, artifacts;

Time dependence of all above factors and conditions.

The term “an imaging area” of a sample holder refers to an area of thesample holder that is to be imaged by an imager.

In certain embodiments, the first image and the second images are morethan one image.

The term “geometric property” of one or more monitoring marks refers tothe shape, size, distance, relative position, total number, numberdensity, area density, rotation, symmetry, periodicity, etc.

The term “optical property” of one or more monitoring marks refers tothe transmission, absorptance, reflectance, fluorescence, scattering,phase contrast, polarization, color spectra, diffusion, phase change,brightness, intensity contrast, Raman scattering, nonlinear harmoniclight generation, electroluminescence, radiation, IR spectra,spontaneous emission, stimulated emission, etc.

The term “an imaging area” of a sample holder refers to an area of thesample holder that is to be imaged by an imager.

In certain embodiments, the spacing between two monitoring makers is 1um, 2 um, 3 um, 5 um, 10 um, 50 um, 100 um, 200 um, 500 um, 1000 um,5000 um or in a range between any of the two values.

In certain embodiments, the preferred spacing between two monitoringmakers is 10 um, 50 um, 100 um, 200 um, or in a range between any of thetwo values.

In certain embodiments, the spacing between two monitoring makers is10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% or in a range betweenany of the two values of the lateral dimension of imaging area.

In certain embodiments, the preferred spacing between two monitoringmakers is 30%, 50%, 80%, 90% or in a range between any of the two valuesof the lateral dimension of imaging area.

In certain embodiments, the average size monitoring makers is 1 um, 2um, 3 um, 5 um, 10 um, 50 um, 100 um, 200 um, 500 um, 1000 um, or in arange between any of the two values.

In certain embodiments, the preferred average size monitoring makers is5 um, 10 um, 50 um, 100 um, or in a range between any of the two values.

In certain embodiments, the average size monitoring makers is 1%, 10%,20%, 30%, 40%, 50%, 60%, 70%, 80%, or in a range between any of the twovalues of the size of imaging area.

In certain embodiments, the preferred average size monitoring makers is1%, 10%, 20%, 30%, or in a range between any of the two values of thesize of imaging area.

Traditional image-based assays may employ a high-precision microscopeequipped with an imaging system to capture the images of the assays.These types of optical microscopes may be subject to the objective lensscaling rule according to which the field-of-view (FoV) is inverselyproportional to the magnification power. According to the rule, theimage for assaying with higher magnification power has a smaller FoV,and the one with larger FoV has less magnification power.

These high-precision microscopes are typically expensive and bulky.Thus, they are commonly used in a lab environment handled by experthuman operators (e.g., a pathologist). In contrast, image-based assayswhere the images are captured by cameras already existing in mass mobiledevices (e.g., smart phones) may provide a low-cost solution for areassuch as low-cost collection and analysis of blood samples of the public.This may be particularly useful in certain healthcare situations, suchas point-of-care (POC), where a large amount of assaying needs to beprocessed quickly and economically. However, the captured assay imagesusing the low-cost mobile device tend to be low-quality containingvariations caused by the wide range of imaging systems from the mobiledevices employed to capture the images. Factors such as the resolutionsand magnifying powers of different cameras and the distortions from theimaging systems or lenses behind the cameras can all vary widely,thereby severely limiting the accuracy and reliability of theimage-based assaying based on these mobile devices.

Therefore, there is a need to improve the low-quality images inimage-based assaying captured by such devices to a level that iscomparable to those captured by high-precision imaging systems such asmicroscopes used in a lab environment. The improved image-based assayusing such devices may overcome the limitation of objective lens scalingrule and provide accurate image-based assaying, especially usinglow-cost commodity optical components for medical, healthcare, POC(point-of-care), chemistry or biology generally.

In this disclosure, the term “System L” may refer to a low-quality imagesystem (e.g. due to inconsistence in lighting/camera optics/imagingsystem sensor array (e.g. photodetector array), spherical distortions,high noise level in captured images, lower magnification, lowerresolutions etc.).

The term “System H” refers to an high-quality image system that meetsthe current regulations for deployment in commercial settings. TheSystem H commonly possesses a superior image quality compared to SystemL (e.g., in terms of a top line optical system with professionallighting/camera optics/imaging system sensor array, low noise level incaptured images, higher magnification power, higher resolutions etc.).

In some embodiments, each of the factors such as light source, optics,and imaging system may also vary across different Systems L. Thisvariation may be caused by lack of calibrations among different typesand/or individual Systems L. The statistical distributions of thesevariations among different Systems L may also vary. To improve theperformance of Systems L, machine learning models may be employed tocompensate the variations in a specific System L. The machine learningmodel such as a neural network model may be customly trained for eachindividual System L. The customly-rained machine learning model can thenbe used to process the images captured by the corresponding individualSystem L during assaying. Although the customly-rained machine learningmodel may improve the performance of each individual System L, it is notsuitable for deployment in a mass market because such training of eachindividual System L (also termed “device” or “assay device”) isinefficient, time consuming, and expensive, and is therefore notpractical in real world applications.

To overcome above-identified and other deficiencies in the currentimplementations, embodiments of the present disclosure provide technicalsolutions that may include a sample holder fabricated with a standard ofpatterned structural elements at predetermined positions, and an imageenhancement method that may produce high quality image-based assays bymitigating the variations caused by individual Systems L based on thestandard including patterned structure elements fabricated in the sampleholder. The image enhancement method may include employment of a machinelearning model and training the machine learning model in the context ofassay images of samples contained in such sample holder utilizing theproperties of the standard. The properties of the standard may includethe geometric properties and/or the material properties of the patternedstructural elements.

Embodiments of the disclosure may provide a method to train a machinelearning model for image-based assays. The method may include receivinga first image, captured by a first imaging system, of a sample holdercontaining a sample, wherein the sample holder is fabricated with astandard of patterned structural elements at predetermined positions,identifying a first region in the first image based on locations of oneor more structural elements of the patterned structural elements in thefirst image, determining a spatial transform associated with the firstregion based on a mapping between the locations of the one or morestructural elements in the first image and predetermined positions ofone or more structural elements in the sample holder, applying thespatial transform to the first region in the first image to calculate atransformed first region, and training the machine learning model usingthe transformed first image.

Embodiments of the disclosure may further include a method to processassay images using the machine learning model trained using theabove-identified method. The method for converting an assay image usingthe machine learning model may include receiving a first image, capturedby a first imaging system, of a sample holder containing a sample,wherein the sample holder is fabricated with a standard of patternedstructural elements at predetermined positions, identifying a firstregion in the first image based on locations of one or more structuralelements of the patterned structural elements in the first image,determining a spatial transform associated with the first region basedon a mapping between the locations of the one or more structuralelements in the first image and predetermined positions of one or morestructural elements in the sample holder, applying the spatial transformto the first region in the first image to calculate a transformed firstregion, and applying the machine learning model to the first image togenerate a second image.

Thus, embodiments of the disclosure provide a system and method that mayeliminate or substantially eliminate, using the standard of the sampleholder exhibiting in an assay image as reference points, variations inthe assay image captured by System L. The embodiments may includeconverting the image into regions in a true dimension space based on thepatterned structural elements in the standard, and further train amachine learning model using these regions in the true dimension space.In this way, embodiments of the disclosure may use such trained machinelearning model to enhance images produced by Systems L to a qualitylevel comparable to those produced by Systems H with little impacts bythe variations caused by individual Systems L.

FIG. 4 illustrates an image-based assay system 1 that may capture assayimages and process the images for training a machine learning modelaccording to an embodiment of the disclosure. Referring to FIG. 4 ,image-based assay system 1 may include a computing system 2, an imagingsystem 3 (e.g., the camera of a smart phone with an adapter), and asample holder device 4. The imaging system 3 may include a built-inlight source, a lens, and an imaging system (e.g., a photodetectorarray). The imaging system 3 may be associated with a computing device(e.g., a mobile smart phone) for capturing assay images. Each imagingsystem 3 associated with an individual computing device may have its ownindividual variations compared with other imaging systems.

The computing system 2 as shown in FIG. 4 can be a standalone computeror a networked computing resource implemented in a computing cloud.Computing system 2 may include one or more processing devices 102, astorage device 104, and an interface device 106, where the storagedevice 104 and the interface device 106 are communicatively coupled toprocessing devices 102.

A processing device 102 can be a hardware processor such as a centralprocessing unit (CPU), a graphic processing unit (GPU), or anaccelerator circuit. Interface device 106 can be a display such as atouch screen of a desktop, laptop, or smart phone. Storage device 104can be a memory device, a hard disc, or a cloud storage 110 connected tocomputing system 2 through a network interface card (not shown).Processing device 102 can be a programmable device that may beprogrammed to implement a machine learning model 108. The implementationof machine learning model 108 may include the training the model and/orthe application of the trained model to image-based assay data.

Imaging system 3 in image-based assay system 1 can be the imaging systemof a System L. For example, imaging system 3 may include the build-inimage sensing photodetector array of a smart phone that is available ina consumer marketplace. Image-based assay system 1 may also include asample holder device 4 for holding a sample 6 (e.g., a biological sampleor a chemical sample) therein. Sample holder device 4 can be a QMAX cardthat is described in detail in International Application No.PCT/US2016/046437. Sample holder device 4 may include two plates that,when in a close configuration, are parallel to each other, wherein atleast one of the two plates is transparent Imaging system 3 may be usedto capture an assay image of the sample 6 contained in sample holderdevice 4 through the transparent plate so that computing system 2 mayanalyze the sample based on the assay image.

The assay images captured by imaging system 3 of a System L aretypically low-quality and are not suitable for further analysis by anexpert human operator or by a computer analysis program. The low qualitymay be reflected in the high level of noise, distortions, and lowresolution in the captured images. A machine learning model 108 may beused to enhance the quality of the assay images. However, as discussedabove, the machine learning model 108 may be ineffective when directlyapplied to assay images because the characterization parameters (e.g.,FoV, magnifying powers, or resolutions) of the imaging systems 3 used tocapture the assay images may vary in a wide range. To mitigate thevariations among imaging systems 3 of different Systems L, in oneembodiment, the sample holder device 4 may be fabricated with a standard5 of patterned structural elements. These patterned structural elementsin standard 5 are precisely-fabricated on an inner surface of at leastone plate of the sample holder device 4. Thus, the positions of thesepatterned structural elements are precise and consistent in the truedimension space on the inner surface. In one embodiment, the structuralelements may have different optical characteristics than sample 6. Thus,these structural elements when captured along with sample 6 may providereference points for correcting the variations caused by differentimaging systems 3.

In one embodiment, the structural elements can be pillarsperpendicularly fabricated on an inner surface of the sample holderdevice 4. The fabrication of these pillars may include grow a nanomaterial on the inner surface. Each pillar may have thethree-dimensional shape of a cylinder that includes a certain height,and a cross-section having a certain area and a certain two-dimensionalshape to make the cross-section detectable from the image of the sampleholder device. Sample holder device 4 may include a first plate and asecond plate. These pillars may be precisely fabricated at predeterminedpositions on the inner surface of first plate or the second plate, wherethe inner surfaces of the first and second plates are the ones that faceeach other when the sample holder device 4 is in the closeconfiguration. Sample 6 may be provided on the inner surface of thefirst plate or on the inner surface of the second plate when the sampleholder device 4 is at the open configuration. After sample holder device4 is closed so that sample 6 is enclosed within the holder andsandwiched between the first and second plates, sample holder device 4may be inserted into an adapter device (not shown) that is mounted onthe computing device associated with imaging system 3. The adapterdevice may hold the sample holder device 4 to stabilize the relativeposition between the sample holder device 4 and imaging system 3 so thatthe imaging system 3 can be activated to capture the assay image 7. Thecaptured assay image 7 in its digital form may include pixelsrepresenting the sample and the cross-section areas of the pillars. Thecaptured assay image 7 may be uploaded to a database system 110 to beused in training machine learning model 108 or analyzed using thetrained machine learning model 108.

The first or second plate may be made from transparent materials. Thus,imaging system 3 with assistance of a light source provided in theadapter may capture an assay image 7 containing pixels representing boththe cross-sections of pillars 5 and sample 6. Assay image 7 may includedistortions (both linear and non-linear) caused by many factorsassociated with Systems L. The factors may include the imprecisecoupling between the sample holder device 4 and the adapter, and thecharacteristic variations inherently existing in imaging systems 3.These distortions if not corrected may adversely impact the performanceof machine learning model 108.

Embodiments of the disclosure may use the positions of pillars 5 todetermine and correct these distortions. In one embodiment, pillars 5are fabricated at predetermined position on an inner surface. Thepositions of these pillars may be located according to a pattern. Forexample, the positions of these pillar may form a rectangular array thathas a horizontal periodicity and a vertical periodicity in the sensethat the horizontal distance (dx) between any two adjacent pillars isthe same and the distance (dy) between any two adjacent pillars is thesame in the true dimension space. Because the positions of these pillarsare predetermined during the manufacture of the sample holder device 4,the detected positions of these pillars in assay image 7 may be used todetermine and correct distortions therein.

Embodiments of the disclosure may include a method 10 to train machinelearning model 108 for enhancing image-based assays. Processing devices102 may be configured to perform method 10. At 112, processing devices102 may receiving a first image, captured by a first imaging system, ofa sample holder containing a sample, wherein the sample holder isfabricated with a standard of patterned structural elements atpredetermined positions. The first image can be an assay image 7captured by imaging system 3, the first image including pixelsrepresenting pillars 5 and sample 6. The sample 5 can be a biologicalsample such as a drop of blood or a chemical sample. As discussed above,the standard of patterned structural elements may include pillarsperpendicularly fabricated on an inner surface of a plate of the sampleholder device 4. The predetermined positions of these pillars may form arectangular array with pillar areas separated by uniform distances inthe horizontal and vertical directions.

At 114, processing devices 102 may identify a first region in the firstimage based on locations of one or more structural elements of thepatterned structural elements in the first image. In this regard,processing devices 102 may first detect, in assay image 7, the locationsof the patterned structural elements in the first image. Because theoptic property (e.g., transparency) and shape of the cross-sections ofpillars are designed to be different than that of the analyte in thesample, the pillars can be differentiated from the sample in assay image7. Embodiments of the disclosure may include any suitable image analysismethods for detecting the locations of the pillars. The methods todetect the locations of pillars may be based on the pixel intensityvalues or morphological properties of regions.

In one embodiment, a second machine learning model may be trained fordetecting locations of pillars in assay image 7. The second machinelearning model can be a neural network such as the RetinaNet which isone-stage object detection model suitable for detecting dense and smallscale objects. In a forward propagation of the training, training assayimages are fed into the second machine model to generate a result imageincluding detected regions of pillars. The result image includingdetected regions of pillars may be compared with regions of pillarslabeled by a human operator. In a backward propagation of the training,parameters of the second machine learning model may be adjusted based onthe difference between the detected regions and the labeled regions. Inthis way, the second machine learning may be trained for the detectionof pillars in assay images.

Embodiments of the disclosure may include detecting regionscorresponding to the pillars in the assay image 7 using the trainedsecond machine learning model. Although the application of the trainedsecond machine learning model to assay image 7 may generate betterpillar detection results than other approaches, the detection resultsmay still include missed detections and false detections of pillars.Embodiments of the disclosure may further include a detection correctionstep to identify missed detection and remove false detection of thepillars. In one embodiment, processing devices 102 may perform thedetection correction step based on the periodic distribution pattern ofpillars. For any position in the periodic distribution pattern thatmisses a corresponding pillar, processing devices 102 may insert acorresponding pillar in the detection results based on the horizontaland/or vertical periodicity of the pattern; for any pillar that is notlocated at a position in the periodic distribution pattern, processingdevices 102 may determine the pillar as a false alarm based on thehorizontal and/or vertical periodicity of the pattern and remove it fromthe detection results.

Based on the locations of pillars in the assay image 7, processingdevices 102 may further partition the first image into regions, whereineach of the regions is defined by detected pillars in assay image 7.When the pillars are arranged according to a rectangular array in thetrue dimension space of the sample holder device 4, assay image 7 may bepartitioned into regions 8 that each is defined by four adjacent pillarsat the four corners. For example, a region may be defined by four linesdrawn from centers of the pillar areas. However, due to distortionsexisting in imprecise coupling between the sample holder device 4 andthe adapter and in imaging system 3, each region may not be arectangular region corresponding to its physical shape in the truedimension space as fabricated on the inner surface of sample holderdevice 4. Instead of a rectangular, each region may be warped into aquadrilateral due to these distortions. Further, the distortions can benon-uniform across the whole assay image 7 (e.g., due to the limitedFoV), resulting in different warping effects for different regions.

Embodiments of the disclosure may mitigate the distortions by projectingeach region back into the true dimension space. Referring to FIG. 4 , at116, processing devices 102 may determine a spatial transform associatedwith the first region based on a mapping between the locations of theone or more structural elements in the first image and predeterminedpositions of one or more structural elements in the sample holder. Eachquadrilateral region defined by four pillar areas at four corners maycorrespond to a rectangular in the true dimension space of sample holderdevice 4 defined by four corresponding pillars. Thus, the parameters ofthe spatial transform may be determined based on the mapping between thefour detected pillar areas in assay image 7 and the four pillars in thetrue dimension space. In one embodiment, the spatial transform can be ahomographic transform (perspective transform) that may map thequadrilateral plane defined by the four detected pillar areas to therectangular plane of its true physical shape defined by the fourcorresponding pillars, where each pillar area may be represented by acenter of the area, and each pillar may be represented by a center ofits cross-section. Processing devices 102 may determine a respectivehomographic transform for each region in assay image 7. Thus, processingdevice 102 may mitigate the distortions associated with each regionusing the homographic transform associated with the region.

At 118, processing devices 102 may apply the spatial transform to thefirst region in the first image to calculate a transformed first region.Processing devices 102 may apply the determined spatial transform topixels in each region to transform the region to the true dimensionspace, thereby substantially eliminating the distortions associated withthe region. In this way, embodiments of the disclosure may usepredetermined positions of pillars in the true dimension space definedon an inner surface of the sample holder device to mitigate orsubstantially remove the distortions of a System L.

At 120, processing device 102 may train the machine learning model 108using the transformed first image. Machine learning model 108 may betrained to enhance assay images captured by Systems L to the qualitylevel comparable to those captured by Systems H. The quality may bereflected in terms of noise level, distortion, and/or the resolutions.The training dataset may include the partitioned regions that have beentransformed into the true dimension space. Thus, the training dataset isless impacted by the distortions caused by Systems L. In one embodiment,the training dataset may include transformed regions from multiple assayimages captured by imaging systems of Systems L.

For the training purpose, the training dataset may also include thecorresponding regions in assay images captured by Systems H. Assayimages captured by Systems H are high quality and suitable for analysisby an expert human operator (e.g., a clinical pathologist) or by acomputer assay analysis program. To construct the training dataset, foreach assay image 7 captured by a System L of a sample holder device 4containing sample 6, the System H may also capture a high-quality assayimage. Processing devices 102 may similarly partition the high-qualityassay image into regions defined by pillars.

FIG. 5 depicts a flow diagram of a method 200 to prepare thehigh-quality assay images in the training dataset according to anembodiment of the disclosure. Method 200 may be performed by processingdevices 102. At 202, processing devices 102 may receive a second imageof the sample holder captured by a second imaging system, wherein thefirst image is captured at a first quality level and the second image iscaptured at a second quality level which is higher than the firstquality level. As discussed above, subsequent to capturing the firstassay image of the sample holder device 4 using the imaging system 3 ofa System L, a second assay image of the same sample holder device 4 maybe captured using a System H. The second assay image may have a higherquality than the corresponding first assay image. The higher quality maybe reflected in a higher resolution, a lower noise level, and/or a lessdistortion. The second assay image may be at a level that can bedirectly used in a lab environment for an expert human operator toanalyze the content of the sample using the second assay image. In oneembodiment, the second assay image may be captured using a microscope ina controlled environment with proper lighting and calibrated optics sothat the captured second assay image may be less impacted by distortionscompared to the first assay image captured by a System L. Thehigh-quality second assay image may have been stored in database system110 after capturing so that processing devices 102 may retrieve thesecond assay image from the database.

At 204, processing devices 204 may partition the second image intoregions, wherein each of the regions in the second image is defined byfour structural elements at four corners of the corresponding region inthe second image. The second assay image may be partitioned in a mannersimilar to the partitioning of the corresponding first assay image. Thepartitioning of the second assay image may include detecting pillarareas in the second assay image, and correcting, based on theperiodicity of the pillar locations, any missing pillar areas and/orfalse pillar areas. Each region in the second assay image may be aquadrilateral that is defined by four adjacent pillar areas at fourcorners.

Based on the locations of detected pillars in the assay image,processing devices 102 may further partition the first image intoregions, wherein each of the regions is defined by detected pillars inthe assay image. When the pillars are arranged according to arectangular array in the true dimension space of the sample holderdevice 4, the second assay image may also be partitioned into regionsthat each is defined by four adjacent pillar areas at the four corners.However, due to distortions existing in imprecise coupling between thesample holder device 4 and the adapter and in imaging system 3, eachregion may not be a rectangular region corresponding to its physicalshape in the true dimension space as fabricated on the inner surface ofsample holder device 4. Similar to the processing of the first assayimage, the distortion associated with each region in the second assayimage may be corrected using a spatial transform.

At 206, processing devices 102 may determine a second spatial transformassociated with a region in the second image based on a mapping betweenthe locations of the four structural elements at the four corners of thecorresponding region in the second image and the four predeterminedpositions of the four structural elements in the sample holder. In oneembodiment, each quadrilateral region defined by four detected pillarareas in the second assay image may correspond to a rectangular in thetrue dimension space of sample holder device 4 defined by fourcorresponding pillars. Thus, the parameters of the second spatialtransform may be determined based on the mapping between the fourdetected pillar areas in the second assay image and the four pillars inthe true dimension space. In one embodiment, the spatial transform canbe a homographic transform (perspective transform) that may map thequadrilateral plane defined by the four detected pillar areas in thesecond assay image to the rectangular plane defined by the fourcorresponding pillars in the sample holder. In another embodiment,instead of determining a respective second spatial transform for eachregion in the second assay image, processing devices 102 may determine aglobal second spatial transform for all regions in the second assayimage. This is possible because the second assay image captured by aSystem H may include a more uniform distortion across all regions due tothe high quality of the imaging system of System H. Thus, a globalsecond spatial transform may be sufficient to correct the uniformdistortion. The global second spatial transform may be determined usingthe pillar areas associated with one of the regions in the second assayimage. Alternatively, the global second spatial transform may bedetermined as an average of second spatial transforms associated withmultiple regions in the second assay image.

At 208, processing devices 102 may apply the second spatial transform toeach of the regions in the second image to calculate a correspondingtransformed region in the second image. The second spatial transform canbe a global transform or a local transform. The application of thesecond spatial transform may transform the regions in the second assayimage to the true dimension space on the inner surface of the samplerholder device, and thus mitigate the distortions associated with thesecond assay image. For each of first assay images in the trainingdataset, processing devices 102 may prepare a corresponding second assayimage. Processing devices 102 may place the transformed regions in thesecond assay images in the training dataset.

At 210, processing devices 102 may train the machine learning modelusing each of transformed regions in the first image and each oftransformed regions in the second image. The machine learning model 108may be trained using the training dataset including the transformedregions of the first assay images and their corresponding regions of thesecond assay images, where both the transformed regions of the firstassay images and their corresponding regions of the second assay imagesare mapped into the true dimension space to mitigate the distortionsexisting in Systems L and Systems H. In this way, the individual systemvariations are substantially removed, and the regions of the first assayimages and regions of the second assay images are mapped to a commontrue dimension space using the standard information embedded in theassay images.

Machine learning model 108 is a model that is built from training dataincluding examples. In this disclosure, the examples include regions ofthe first assay images and their corresponding regions in the secondassay image in the training dataset. These regions in the trainingdataset are already corrected for the distortions by transforming intothe true dimension space defining the positions of the pillars on theinner surface of the sample holder device 4. Embodiments may use thetrained machine learning model 108 to enhance the assay images capturedby Systems L to a quality level comparable to those captured by SystemsH.

Machine learning model 108 can be any suitable model including but notlimited to deep neural network (DNN), convolutional neural network(CNN), recurrent neural network (RNN), generative adversarial network(GAN), graph neural network (GNN) etc. All of these models may includeparameters that can be adjusted during training based on trainingdataset or examples.

The mapping from assay images of Systems L to assay images of Systems Hcan belong to a class of image-to-image transformation problems. Thetraining of the machine learning model used in image-to-imagetransformation may require a large quantity of perfectly matched pairedexamples from the source images (assay images captured by Systems L) tothe target images (assay images captured by Systems H). In the contextof assay images, the sample contained in the sample holder is commonly atype of liquid in which the minute analytes may move constantly betweenthe images taken by different imaging systems (e.g., System L and thenSystem H). Therefore, it is not practical if not impossible to constructa sufficiently large training dataset containing perfectly matched pairsof low-quality and high-quality assay image examples for training amachine learning model used for image transformation. To overcome thispractical problem in the transformation of assay images, embodiments ofthe present disclosure employ the cycle generative adversarial network(CycleGAN) model that may be trained using unpaired assay images.Although the regions partitioned from the low-quality assay image arenot paired with the corresponding regions partitioned from thehigh-quality assay image due to the movement of the analyte in thesample and other factors, the pillar areas at the corners of regions inthe low-quality assay image are matched to the pillar areas at thecorners of the high-quality assay image because they are physicallyfixed in the true dimension space during the precise fabrication of thesample holder. The matched pillar areas embedded in the assay images inthe present disclosure provide registered landmarks information andadditional constraints that help further improve the training of theCycleGAN and the fidelity of the transformed image for assayingpurposes.

The CycleGAN model is composed of a forward GAN and a backward GAN. FIG.6 illustrates the construction of a CycleGAN model 300 used for assayimage-to-image translation according to an embodiment of the disclosure.Referring to FIG. 6 , CycleGAN model 300 may include a forward GAN 302and a backward GAN 304. Forward GAN 302 like a typical GAN model mayinclude a generator 306A and a discriminator 306B; similarly, backwardGAN 304 may also include a generator 308A and a discriminator 308B. Eachof generators 306A, 308A and discriminators 306B, 308B can be a neuralnetwork (e.g., a multi-layered convolutional neural network). Generators306A, 308A may convert an input assay image in a first domain (e.g.,having a first quality or first resolution) into an output assay imagein a second domain (e.g., having a second quality of second resolution).

Generator 306A may convert assay images 310 in a first image domain(domain L) into generated assay images in a second image domain (domainH). For example, generator 306A may convert low-resolution assay imagescaptured by Systems L into generated high-resolution assay images with asame resolution as those captured by Systems H. Generator 308A mayconvert assay images 312 in the domain H into generated assay images indomain L For example, generator 308A may convert high-resolution assayimages captured by Systems H into generated low-resolution assay imagesin domain L. Discriminator 306B may compare the generated assay imagesin domain H with real assay images in domain H to output a firstgenerator loss function and a first discriminator loss function. Thefirst generator loss function may indicate whether the generated assayimages in domain H belong to domain H (or “real”) or not (or “fake”).The first discriminator loss function (not shown) may indicate whetherthe discriminator 306B makes the correct classification of “real” or“fake.” Discriminator 308B may compare the generated assay images indomain L with real assay images in domain L to output a second generatorloss function and a second discriminator loss function. The secondgenerator loss function may indicate whether the generated assay imagesin domain L belong to domain L (or “real”) or not (or “fake”). Thesecond discriminator loss function (not shown) may indicate whether thediscriminator 308B makes the correct classification of “real” or “fake.”

During training, the first discriminator loss function may be used in abackpropagation to train the discriminator 306B, and the firstdiscriminator loss function may be used in a backpropagation to trainthe discriminator 308B. The parameters of generator 306 may be adjustedin a backpropagation based on the first generator loss function fromdiscriminator 306B so that generator 306A may produce generated assayimages in domain H considered by discriminator 306B as “real.”Similarly, the parameters of generator 308A may be adjusted in abackpropagation based on the second generator loss function fromdiscriminator 308B so that generator 308A may produce generated assayimages in domain L considered by discriminator 306B as “real.”Discriminator 306B, 308B and generator 306A, 308A contained in a GAN302, 304 may be trained in alternative time periods. For example,discriminator 306B, 308B may be trained in several epochs in step 1, andthen generator 306A, 308A may be trained in subsequent several epochs instep 2. The steps 1 and 2 may be repeated alternatively during traininguntil each of the GANs 302, 304 converges.

In one optional embodiment, the CycleGAN 300 may require the cycleconsistency. The cycle consistency requires that the combinedtransformation of generator 306A and generator 308A results in anidentity cycle mapping. The identity mapping means that generator 306Amay convert an input assay image to a generated assay image, andgenerator 308A may convert the generated assay image back to theoriginal input assay image. The cycle consistency requirement may allowCycleGAN 300 to work using unpaired images to train the imagetransformation model. Further, embodiments of the present disclosureprovide additional constraints such as the registered landmarkinformation of the pillar areas to output high-fidelity transformedassay images and position correspondence of the transformed assay imagesfor assaying purposes.

In the context of the present disclosure, low-resolution images 310 mayinclude regions partitioned based on detected pillar areas from assayimages captured by imaging system 3, and high-resolution images 312 mayinclude the corresponding regions partitioned based on detected pillarareas from assay images captured by a high-quality imaging system builtin a microscope, where all the assay image regions have been transformedinto the true dimension space to mitigate the distortions. In trainingof the forward GAN 302, a first assay image region from low-resolutionimages 310 may be provided to generator 306A to produce a generatedfirst region in the high resolution domain. Discriminator 306B maycompare the generated first region with the corresponding first regionassay image in high-resolution assay images 312 to produce the firstgenerator loss function and first discriminator loss function. The firstdiscriminator loss function may be used in a backpropagation to traindiscriminator 306B, and the first generator loss function may be used ina backpropagation to train generator 306A. Each assay image region inlow-resolution images 310 may be similarly used to train generator 306Aand discriminator 306B of the forward GAN 302. In training of thebackward GAN 304, a second assay image region from high-resolutionimages 312 may be provided to generator 308A to produce a generatedsecond region in the low resolution domain. Discriminator 308B maycompare the generated second region with the corresponding second regionassay image in low-resolution assay images 310 to produce the secondgenerator loss function and second discriminator loss function. Thesecond discriminator loss function may be used in a backpropagation totrain discriminator 308B, and the second generator loss function may beused in a backpropagation to train generator 308A. Each assay imageregion in high-resolution images 312 may be similarly used to traingenerator 308A and discriminator 308B of the backward GAN 304.

In application, the trained CycleGAN 300 may be used to convert an assayimage captured by imaging system 3 to a generated assay image at aquality level comparable to those captured by a microscope. FIG. 7depicts a flow diagram of a method 400 to enhance a low-quality assayimage according to an embodiment of the disclosure. One or moreprocessing devices (e.g., processing devices 102) may perform theoperations of method 400.

At 402, the processing devices may receive a first image, captured by afirst imaging system, of a sample holder containing a sample, where thesample holder is fabricated with a standard of patterned structuralelements at predetermined positions. The first image can be a firstassay image including pixels representing the sample and the structuralelements such as pillars.

At 404, the processing devices may identify a first region in the firstimage based on locations of one or more structural elements of thepatterned structural elements in the first image. The identification ofthe first region may include detecting pillar areas in the first imageand identifying the first region based on four adjacent pillar areas atfour corners. The first region can be a quadrilateral due to distortionsassociated with the first image.

At 406, the processing devices may determine a spatial transformassociated with the first region based on a mapping between thelocations of the one or more structural elements in the first image andpredetermined positions of one or more structural elements in the sampleholder. The spatial transform can be a homographic transform. Theparameters of the spatial transform can be determined based on a mappingof the positions (e.g., the centers) of the four pillar areas at thefour corners of the first region with the positions (e.g., the centersof cross-sections) of the four pillars on the true dimension space on aninner surface of the sample holder device.

At 408, the processing devices may apply the spatial transform to thefirst region in the first image to calculate a transformed first region.The application of the spatial transform to the first region may helpremove the distortions based on the positions of the pillars.

At 410, the processing devices may apply a machine learning model to thetransformed first region in the first image to generate a second region.The machine learning model can be the generator 306A as shown in FIG. 6. In one embodiment, the second image region may have a higherresolution than the first image region. For example, the second imageregion may be at a resolution of microscopic image.

The processing devices may further process each identified region in thefirst image according to steps 402-404 to generate a correspondinghigh-resolution region. Further, processing devices may recombine thesegenerated high-resolution regions to form a second image which is ahigh-resolution version of the first image. The second image may beanalyzed by an expert human operator or by another intelligent computersystem for its contents.

Instead of applying a spatial transform each region of an assay imagecaptured by System L, some embodiments may apply a global spatialtransform to the whole image in preparing the training dataset. FIG. 8depicts a flow diagram of a method 500 for preparing a training datasetfor a machine learning model according to an embodiment of thedisclosure.

At 502, the processing devices may receive a low-quality assay imagecaptured by an imaging system of a system L of a sample holder devicecontaining a sample and a standard of patterned pillars.

At 504, the processing devices may detect pillar locations in thelow-quality assay image, and optionally, detect the orientation of thepattern of the pillar locations. A machine learning model as discussedabove may be used to detect pillar locations in the low-quality assayimage. The orientation of the pattern may be detected based on thehorizontal inter-pillar distance, the vertical inter-pillar distance,the count of pillars in the horizontal direction, and/or the count ofpillars in the vertical direction.

At 506, the processing devices may determine a global spatial transformbased on a mapping between the pillar locations in the low-quality assayimage and the pillar positions in the true dimension space of the sampleholder device. The processing devices may optionally perform otherpre-processing operations such as estimating the field of view (FoV)based the pillar locations in the low-quality image and correcting thelow-quality image based on the estimated FoV.

At 508, the processing devices may partition the low-quality image intoregions, each of the regions being defined by four pillars at fourcorners.

At 510, the processing devices may optionally resize or scale eachregion to a common size (e.g., 256×256 pixels). This resize or rescalemay prepare the data for the calculation using a later machine learningmodel.

At 512, the processing devices may rotate the low-quality images (andthe regions therein) according to the orientation of the pattern of thepillars. This operation is to ensure that all images are compared in thesame pattern orientation.

At 514, the processing devices may optionally convert each pixels to agreyscale pixel (e.g., from RGB color to greyscale). This operation mayfurther reduce the calculation by the machine learning model.

At 516, the processing devices may store the such processed regions oflow-quality images as domain L examples in the training dataset.

The high-quality assay images may be similarly processed.

At 518, the processing devices may receive a high-quality assay imagecaptured by an imaging system of a system H of a sample holder devicecontaining a sample and a standard of patterned pillars.

At 520, the processing devices may detect pillar locations in thehigh-quality assay image, and optionally, detect the orientation of thepattern of the pillar locations. A machine learning model as discussedabove may be used to detect pillar locations in the high-quality assayimage. The orientation of the pattern may be detected based on thehorizontal inter-pillar distance, the vertical inter-pillar distance,the count of pillars in the horizontal direction, and/or the count ofpillars in the vertical direction.

At 522, the processing devices may determine a global spatial transformbased on a mapping between the pillar locations in the high-qualityassay image and the pillar positions in the true dimension space of thesample holder device. The processing devices may optionally performother pre-processing operations such as estimating the field of view(FoV) based the pillar locations in the high-quality image andcorrecting the high-quality image based on the estimated FoV.

At 524, the processing devices may partition the high-quality image intoregions, each of the regions being defined by four pillars at fourcorners.

At 526, the processing devices may optionally resize or scale eachregion to a common size (e.g., 256×256 pixels). This resize or rescalemay prepare the data for the calculation using a later machine learningmodel.

At 528, the processing devices may rotate the high-quality images (andthe regions therein) according to the orientation of the pattern of thepillars. This operation is to ensure that all images are compared in thesame pattern orientation.

At 530, the processing devices may optionally convert each pixels to agreyscale pixel (e.g., from RGB color to greyscale). This operation mayfurther reduce the calculation by the machine learning model.

At 532, the processing devices may store the such processed regions ofhigh-quality images as domain H examples in the training dataset. Thus,the training dataset may be constructed.

While embodiments of the disclosure are described in the context ofimage-based assaying using a sample holder device fabricated with astandard of patterned structural elements as landmark references in theassay images, the system and method of machine learning are readilyapplicable to other types of material imaging where the material beingimaged may be morphed from a state of well-defined shape (e.g., crystal)to an amorphous state, and the imaging system used to capture the imageof the amorphous material is imperfect. In some implementations, amachine learning model may be trained to learn the mapping from thematerial image of an amorphous material captured by an imperfect imagingsystem to the material image of a crystalline material captured by aperfect imaging system. However, such a direct training of the machinelearning model requires a huge amount of training examples (e.g., in theorder 10,000 or more), which is impractical due to the high cost.

FIG. 10A illustrates the mapping between a crystalline structure 702 andan amorphous structure 704. A crystalline structure 702 represents amaterial structure that has a certain pattern (e.g., periodic atomarrangement). In contrast, an amorphous structure 704 represents amaterial structure that has no pattern. In a traditional test paradigm(e.g., precision protocol paradigm (PPP)), the image of the crystallinestructure 702 may be captured by a high precision instrument (e.g., anelectronic microscope) by a professional human operator while thecrystalline structure 702 is in perfect shape. In an intelligent testparadigm, the image of the amorphous structure 704 may be captured by animprecise instrument (e.g., an imaging system of a smart phone) by anon-professional human operator. It is an objective of the intelligenttest paradigm to map the amorphous structure 704 to the crystallinestructure 702 using a machine learning model. As discussed above, thetraining of the machine learning model requires a huge amount oftraining data which are not readily available. Additionally, the directmapping from the amorphous structure 704 to the crystalline structure702 using such trained machine learning model may result in machinelearning artifacts in the material image. Therefore, there is a need toreduce to the requirement for a large amount of training data and toimprove the image fidelity.

Instead of trying to train a machine learning model that maps directlyfrom the amorphous structure 704 to the crystalline structure 702,embodiments of the disclosure may introduce an intermediate structure tothe crystalline structure and the amorphous structure during theconstruction of the training data. The machine learning model may betrained using the intermediate structure, thus reducing the requirementfor training data and improving the image fidelity. FIG. 10B illustratesthe mapping between a crystalline structure 704 injected with a metastructure and an amorphous structure 706 injected with the metastructure according to an embodiment of the disclosure. Meta structuremay include elements with prominent physical characteristics (such asshapes and optical properties) that can easily distinguish theseelements from the carrier crystalline/amorphous structures in a materialimage. Thus, when the training data contain material images of thecrystalline structures 704 injected with the meta structure and materialimages of the amorphous structure 706 injected with the meta structure,the meta structure can be extracted first from these material images andcan be used as training data to train the machine learning model.Because the physical properties and characteristics of the metastructure are known in advance, the meta structure can be extracted withconfidence from the material images captured in the intelligent testparadigm using imprecise imaging systems by non-professional operators.Also because of the known physical properties and characteristics, thetraining of the machine learning model may require less training dataand result in high-fidelity images.

Embodiments of the disclosure may include a method for training amachine learning model that maps of images of a first material of afirst structure to images of a second material of a second structure.The method includes injecting a meta structure into the first materialand capturing a first image of the first material using a first imagingsystem, injecting the meta structure into the second material andcapturing a second image using a second imaging system, extracting firstpositions of the meta structure from the first image, extracting secondpositions of the meta structure from the second image, and training amachine learning model using the first positions and the secondpositions. The first material includes an amorphous structure, and thesecond material includes a crystalline structure. The method furtherincludes applying the trained machine learning model to map an image ofa third material having the first structure to an image of a fourthmaterial having the second structure.

The sample holder devices as described in this disclosure can be a QMAXcard. Technical details are described in International Application No.PCT/US2016/046437. A QMAX card may include two plates.

I. Plates

In present disclosure, generally, the plates of Compressed RegulatedOpen Flow (CROF) are made of any material that (i) is capable of beingused to regulate, together with the spacers, the thickness of a portionor entire volume of the sample, and (ii) has no significant adverseeffects to a sample, an assay, or a goal that the plates intend toaccomplish. However, in certain embodiments, particular materials (hencetheir properties) ae used for the plate to achieve certain objectives.

In certain embodiments, the two plates have the same or differentparameters for each of the following parameters: plate material, platethickness, plate shape, plate area, plate flexibility, plate surfaceproperty, and plate optical transparency.

(i) Plate Materials. The plates are made a single material, compositematerials, multiple materials, multilayer of materials, alloys, or acombination thereof. Each of the materials for the plate is an inorganicmaterial, am organic material, or a mix, wherein examples of thematerials are given in paragraphs of Mat-1 and Mat-2.

Mat-1: The inorganic materials for the plates include, not limited to,glass, quartz, oxides, silicon-dioxide, silicon-nitride, hafnium oxide(HfO), aluminum oxide (AIO), semiconductors: (silicon, GaAs, GaN, etc.),metals (e.g. gold, silver, coper, aluminum, Ti, Ni, etc.), ceramics, orany combinations of thereof.

Mat-2: The organic materials for the spacers include, not limited to,polymers (e.g. plastics) or amorphous organic materials. The polymermaterials for the spacers include, not limited to, acrylate polymers,vinyl polymers, olefin polymers, cellulosic polymers, noncellulosicpolymers, polyester polymers, Nylon, cyclic olefin copolymer (COC),poly(methyl methacrylate) (PMMA), polycarbonate (PC), cyclic olefinpolymer (COP), liquid crystalline polymer (LCP), polyamide (PA),polyethylene (PE), polyimide (PI), polypropylene (PP), poly(phenyleneether) (PPE), polystyrene (PS), polyoxymethylene (POM), polyether etherketone (PEEK), polyether sulfone (PES), poly(ethylene phthalate) (PET),polytetrafluoroethylene (PTFE), polyvinyl chloride (PVC), polyvinylidenefluoride (PVDF), polybutylene terephthalate (PBT), fluorinated ethylenepropylene (FEP), perfluoroalkoxyalkane (PFA), polydimethylsiloxane(PDMS), rubbers, or any combinations of thereof.

In certain embodiments, the plates are each independently made of atleast one of glass, plastic, ceramic, and metal. In certain embodiments,each plate independently includes at least one of glass, plastic,ceramic, and metal.

In certain embodiments, one plate is different from the other plate inlateral area, thickness, shape, materials, or surface treatment. Incertain embodiments, one plate is the same as the other plate in lateralarea, thickness, shape, materials, or surface treatment.

The materials for the plates are rigid, flexible or any flexibilitybetween the two. The rigid (e.g. stiff) or flexibility is relative to agive pressing forces used in bringing the plates into the closedconfiguration.

In certain embodiments, a selection of rigid or flexible plate aredetermined from the requirements of controlling a uniformity of thesample thickness at the closed configuration.

In certain embodiments, at least one of the two plates are transparent(to a light). In certain embodiments at least a part or several parts ofone plate or both plates are transparent. In certain embodiments, theplates are non-transparent.

(ii) Plate Thickness. In certain embodiments, the average thicknessesfor at least one of the pates are 2 nm or less, 10 nm or less, 100 nm orless, 500 nm or less, 1000 nm or less, 2 um (micron) or less, 5 um orless, 10 um or less, 20 um or less, 50 um or less, 100 um or less, 150um or less, 200 um or less, 300 um or less, 500 um or less, 800 um orless, 1 mm (millimeter) or less, 2 mm or less, 3 mm or less, or a rangebetween any two of the values.

In certain embodiments, the average thicknesses for at least one of theplates are at most 3 mm (millimeter), at most 5 mm, at most 10 mm, atmost 20 mm, at most 50 mm, at most 100 mm, at most 500 mm, or a rangebetween any two of the values.

In certain embodiments, the thickness of a plate is not uniform acrossthe plate. Using a different plate thickness at different location canbe used to control the plate bending, folding, sample thicknessregulation, and others.

(iii) Plate Shape and Area. Generally, the plates can have any shapes,as long as the shape allows a compress open flow of the sample and theregulation of the sample thickness. However, in certain embodiments, aparticular shape can be advantageous. The shape of the plate can beround, elliptical, rectangles, triangles, polygons, ring-shaped, or anysuperpositions of these shapes.

In certain embodiments, the two plates can have the same size or shape,or different. The area of the plates depend on the application. The areaof the plate is at most 1 mm2 (millimeter square), at most 10 mm2, atmost 100 mm2, at most 1 cm2 (centimeter square), at most 5 cm2, at most10 cm2, at most 100 cm2, at most 500 cm2, at most 1000 cm2, at most 5000cm2, at most 10,000 cm2, or over 10,000 cm2, or any arrange between anyof the two values. The shape of the plate can be rectangle, square,round, or others.

In certain embodiments, at least one of the plates is in the form of abelt (or strip) that has a width, thickness, and length. The width is atmost 0.1 cm (centimeter), at most 0.5 cm, at most 1 cm, at most 5 cm, atmost 10 cm, at most 50 cm, at most 100 cm, at most 500 cm, at most 1000cm, or a range between any two of the values. The length can be as longit needed. The belt can be rolled into a roll.

(iv) Plate Surface Flatness. In many embodiments, an inner surface ofthe plates are flat or significantly flat, planar. In certainembodiments, the two inner surfaces are, at the closed configuration,parallel with each other. Flat inner surfaces facilitates aquantification and/or controlling of the sample thickness by simplyusing the predetermined spacer height at the closed configuration. Fornon-flat inner surfaces of the plate, one need to know not only thespacer height, but also the exact the topology of the inner surface toquantify and/or control the sample thickness at the closedconfiguration. To know the surface topology needs additionalmeasurements and/or corrections, which can be complex, time consuming,and costly.

A flatness of the plate surface is relative to the final samplethickness (the final thickness is the thickness at the closedconfiguration), and is often characterized by the term of “relativesurface flatness” is the ratio of the plate surface flatness variationto the final sample thickness.

In certain embodiments, the relative surface is less than 0.01%, 0.1%,less than 0.5%, less than 1%, less than 2%, less than 5%, less than 10%,less than 20%, less than 30%, less than 50%, less than 70%, less than80%, less than 100%, or a range between any two of these values.

(v) Plate Surface Parallelness. In certain embodiments, the two surfacesof the plate is significantly parallel with each other. In certainembodiments, the two surfaces of the plate is not parallel with eachother.

(vi) Plate Flexibility. In certain embodiments, a plate is flexibleunder the compressing of a CROF process. In certain embodiments, bothplates are flexible under the compressing of a CROF process. In certainembodiments, a plate is rigid and another plate is flexible under thecompressing of a CROF process. In certain embodiments, both plates arerigid. In certain embodiments, both plate are flexible but havedifferent flexibility.

(vii) Plate Optical Transparency. In certain embodiments, a plate isoptical transparent. In certain embodiments, both plates are opticaltransparent. In certain embodiments, a plate is optical transparent andanother plate is opaque. In certain embodiments, both plates are opaque.In certain embodiments, both plate are optical transparent but havedifferent optical transparency. The optical transparency of a plate canrefer to a part or the entire area of the plate.

(viii) Surface Wetting Properties. In certain embodiments, a plate hasan inner surface that wets (e.g. contact angle is less 90 degree) thesample, the transfer liquid, or both. In certain embodiments, bothplates have an inner surface that wets the sample, the transfer liquid,or both; either with the same or different wettability. In certainembodiments, a plate has an inner surface that wets the sample, thetransfer liquid, or both; and another plate has an inner surface thatdoes not wet (e.g. the contact angle equal to or larger than 90 degree).The wetting of a plate inner surface can refer to a part or the entirearea of the plate.

In certain embodiments, the inner surface of the plate has other nano ormicrostructures to control a lateral flow of a sample during a CROF. Thenano or microstructures include, but not limited to, channels, pumps,and others. Nano and microstructures are also used to control thewetting properties of an inner surface.

II. Spacers

(i) Spacers' Function. In present invention, the spacers are configuredto have one or any combinations of the following functions andproperties: the spacers are configured to (1) control, together with theplates, the thickness of the sample or a relevant volume of the sample(Preferably, the thickness control is precise, or uniform or both, overa relevant area); (2) allow the sample to have a compressed regulatedopen flow (CROF) on plate surface; (3) not take significant surface area(volume) in a given sample area (volume); (4) reduce or increase theeffect of sedimentation of particles or analytes in the sample; (5)change and/or control the wetting propertied of the inner surface of theplates; (6) identify a location of the plate, a scale of size, and/orthe information related to a plate, or (7) do any combination of theabove.

(ii) Spacer Architectures and Shapes. To achieve desired samplethickness reduction and control, in certain embodiments, the spacers arefixed its respective plate. In general, the spacer can have any shape,as long as the spacers are capable of regulating the sample thicknessduring a CROF process, but certain shapes are preferred to achievecertain functions, such as better uniformity, less overshoot inpressing, etc.

The spacer(s) is a single spacer or a plurality of spacers. (e.g. anarray). Certain embodiments of a plurality of spacers is an array ofspacers (e.g. pillars), where the inter-spacer distance is periodic oraperiodic, or is periodic or aperiodic in certain areas of the plates,or has different distances in different areas of the plates.

There are two kinds of the spacers: open-spacers and enclosed-spacers.The open-spacer is the spacer that allows a sample to flow through thespacer (e.g. the sample flows around and pass the spacer. For example, apost as the spacer.), and the enclosed spacer is the spacer that stopthe sample flow (e.g. the sample cannot flow beyond the spacer. Forexample, a ring shape spacer and the sample is inside the ring.). Bothtypes of spacers use their height to regular the final sample thicknessat a closed configuration.

In certain embodiments, the spacers are open-spacers only. In certainembodiments, the spacers are enclosed-spacers only. In certainembodiments, the spacers are a combination of open-spacers andenclosed-spacers.

The term “pillar spacer” means that the spacer has a pillar shape andthe pillar shape can refer to an object that has height and a lateralshape that allow a sample to flow around it during a compressed openflow.

In certain embodiments, the lateral shapes of the pillar spacers are theshape selected from the groups of (i) round, elliptical, rectangles,triangles, polygons, ring-shaped, star-shaped, letter-shaped (e.g.L-shaped, C-shaped, the letters from A to Z), number shaped (e.g. theshapes like 0 1, 2, 3, 4, . . . to 9); (ii) the shapes in group (i) withat least one rounded corners; (iii) the shape from group (i) withzig-zag or rough edges; and (iv) any superposition of (i), (ii) and(iii). For multiple spacers, different spacers can have differentlateral shape and size and different distance from the neighboringspacers.

In certain embodiments, the spacers can be and/or can include posts,columns, beads, spheres, and/or other suitable geometries. The lateralshape and dimension (e.g., transverse to the respective plate surface)of the spacers can be anything, except, in certain embodiments, thefollowing restrictions: (i) the spacer geometry will not cause asignificant error in measuring the sample thickness and volume; or (ii)the spacer geometry would not prevent the out-flowing of the samplebetween the plates (e.g. it is not in enclosed form). But in certainembodiments, they require some spacers to be closed spacers to restrictthe sample flow.

In certain embodiments, the shapes of the spacers have rounded corners.For example, a rectangle shaped spacer has one, several or all cornersrounded (like a circle rather 90 degree angle). A round corner oftenmake a fabrication of the spacer easier, and in some cases less damageto a biological material.

The sidewall of the pillars can be straight, curved, sloped, ordifferent shaped in different section of the sidewall. In certainembodiments, the spacers are pillars of various lateral shapes,sidewalls, and pillar-height to pillar lateral area ratio. In apreferred embodiment, the spacers have shapes of pillars for allowingopen flow.

(iii) Spacers' Materials. In the present invention, the spacers aregenerally made of any material that is capable of being used toregulate, together with the two plates, the thickness of a relevantvolume of the sample. In certain embodiments, the materials for thespacers are different from that for the plates. In certain embodiments,the materials for the spaces are at least the same as a part of thematerials for at least one plate.

The spacers are made a single material, composite materials, multiplematerials, multilayer of materials, alloys, or a combination thereof.Each of the materials for the spacers is an inorganic material, amorganic material, or a mix, wherein examples of the materials are givenin paragraphs of Mat-1 and Mat-2. In a preferred embodiment, the spacersare made in the same material as a plate used in CROF.

(iv) Spacers' Mechanical Strength and Flexibility. In certainembodiments, the mechanical strength of the spacers are strong enough,so that during the compression and at the closed configuration of theplates, the height of the spacers is the same or significantly same asthat when the plates are in an open configuration. In certainembodiments, the differences of the spacers between the openconfiguration and the closed configuration can be characterized andpredetermined.

The material for the spacers is rigid, flexible or any flexibilitybetween the two. The rigid is relative to a give pressing forces used inbringing the plates into the closed configuration: if the space does notdeform greater than 1% in its height under the pressing force, thespacer material is regarded as rigid, otherwise a flexible. When aspacer is made of material flexible, the final sample thickness at aclosed configuration still can be predetermined from the pressing forceand the mechanical property of the spacer.

(v) Spacers Inside Sample. To achieve desired sample thickness reductionand control, particularly to achieve a good sample thickness uniformity,in certain embodiments, the spacers are placed inside the sample, or therelevant volume of the sample. In certain embodiments, there are one ormore spacers inside the sample or the relevant volume of the sample,with a proper inter spacer distance. In certain embodiments, at leastone of the spacers is inside the sample, at least two of the spacersinside the sample or the relevant volume of the sample, or at least of“n” spacers inside the sample or the relevant volume of the sample,where “n” can be determined by a sample thickness uniformity or arequired sample flow property during a CROF.

(vi) Spacer Height. In certain embodiments, all spacers have the samepre-determined height. In certain embodiments, spacers have differentpre-determined height. In certain embodiments, spacers can be dividedinto groups or regions, wherein each group or region has its own spacerheight. And in certain embodiments, the predetermined height of thespacers is an average height of the spacers. In certain embodiments, thespacers have approximately the same height. In certain embodiments, apercentage of number of the spacers have the same height.

The height of the spacers is selected by a desired regulated finalsample thickness and the residue sample thickness. The spacer height(the predetermined spacer height) and/or sample thickness is 3 nm orless, 10 nm or less, 50 nm or less, 100 nm or less, 200 nm or less, 500nm or less, 800 nm or less, 1000 nm or less, 1 um or less, 2 um or less,3 um or less, 5 um or less, 10 um or less, 20 um or less, 30 um or less,50 um or less, 100 um or less, 150 um or less, 200 um or less, 300 um orless, 500 um or less, 800 um or less, 1 mm or less, 2 mm or less, 4 mmor less, or a range between any two of the values.

The spacer height and/or sample thickness is between 1 nm to 100 nm inone preferred embodiment, 100 nm to 500 nm in another preferredembodiment, 500 nm to 1000 nm in a separate preferred embodiment, 1 um(e.g. 1000 nm) to 2 um in another preferred embodiment, 2 um to 3 um ina separate preferred embodiment, 3 um to 5 um in another preferredembodiment, 5 um to 10 um in a separate preferred embodiment, and 10 umto 50 um in another preferred embodiment, 50 um to 100 um in a separatepreferred embodiment.

In certain embodiments, the spacer height and/or sample thickness (i)equal to or slightly larger than the minimum dimension of an analyte, or(ii) equal to or slightly larger than the maximum dimension of ananalyte. The “slightly larger” means that it is about 1% to 5% largerand any number between the two values.

In certain embodiments, the spacer height and/or sample thickness islarger than the minimum dimension of an analyte (e.g. an analyte has ananisotropic shape), but less than the maximum dimension of the analyte.

For example, the red blood cell has a disk shape with a minim dimensionof 2 um (disk thickness) and a maximum dimension of 11 um (a diskdiameter). In an embodiment of the present invention, the spacers isselected to make the inner surface spacing of the plates in a relevantarea to be 2 um (equal to the minimum dimension) in one embodiment, 2.2um in another embodiment, or 3 (50% larger than the minimum dimension)in other embodiment, but less than the maximum dimension of the redblood cell. Such embodiment has certain advantages in blood cellcounting. In one embodiment, for red blood cell counting, by making theinner surface spacing at 2 or 3 um and any number between the twovalues, a undiluted whole blood sample is confined in the spacing, onaverage, each red blood cell (RBC) does not overlap with others,allowing an accurate counting of the red blood cells visually. (Too manyoverlaps between the RBC's can cause serious errors in counting).

In the present invention, in certain embodiments, it uses the plates andthe spacers to regulate not only a thickness of a sample, but also theorientation and/or surface density of the analytes/entity in the samplewhen the plates are at the closed configuration. When the plates are ata closed configuration, a thinner thickness of the sample gives a lessthe analytes/entity per surface area (e.g. less surface concentration).

(vii) Spacer Lateral Dimension. For an open-spacer, the lateraldimensions can be characterized by its lateral dimension (sometime beingcalled width) in the x and y—two orthogonal directions. The lateraldimension of a spacer in each direction is the same or different.

In certain embodiments, the ratio of the lateral dimensions of x to ydirection is 1, 1.5, 2, 5, 10, 100, 500, 1000, 10,000, or a rangebetween any two of the value. In certain embodiments, a different ratiois used to regulate the sample flow direction; the larger the ratio, theflow is along one direction (larger size direction).

In certain embodiments, the different lateral dimensions of the spacersin x and y direction are used as (a) using the spacers as scale-markersto indicate the orientation of the plates, (b) using the spacers tocreate more sample flow in a preferred direction, or both.

In a preferred embodiment, the period, width, and height.

In certain embodiments, all spacers have the same shape and dimensions.In certain embodiments, each of the spacers have different lateraldimensions.

For enclosed-spacers, in certain embodiments, the inner lateral shapeand size are selected based on the total volume of a sample to beenclosed by the enclosed spacer(s), wherein the volume size has beendescribed in the present disclosure; and in certain embodiments, theouter lateral shape and size are selected based on the needed strengthto support the pressure of the liquid against the spacer and thecompress pressure that presses the plates.

(viii) Aspect Ratio of Height to the Average Lateral Dimension of PillarSpacer. In certain embodiments, the aspect ratio of the height to theaverage lateral dimension of the pillar spacer is 100,000, 10,000,1,000, 100, 10, 1, 0.1, 0.01, 0.001, 0.0001, 0, 00001, or a rangebetween any two of the values.

(ix) Spacer Height Precisions. The spacer height should be controlledprecisely. The relative precision of the spacer (e.g. the ratio of thedeviation to the desired spacer height) is 0.001% or less, 0.01% orless, 0.1% or less; 0.5% or less, 1% or less, 2% or less, 5% or less, 8%or less, 10% or less, 15% or less, 20% or less, 30% or less, 40% orless, 50% or less, 60% or less, 70% or less, 80% or less, 90% or less,99.9% or less, or a range between any of the values.

(x) Inter-Spacer Distance. The spacers can be a single spacer or aplurality of spacers on the plate or in a relevant area of the sample.In certain embodiments, the spacers on the plates are configured and/orarranged in an array form, and the array is a periodic, non-periodicarray or periodic in some locations of the plate while non-periodic inother locations.

In certain embodiments, the periodic array of the spacers has a latticeof square, rectangle, triangle, hexagon, polygon, or any combinations ofthereof, where a combination means that different locations of a platehas different spacer lattices.

In certain embodiments, the inter-spacer distance of a spacer array isperiodic (e.g. uniform inter-spacer distance) in at least one directionof the array. In certain embodiments, the inter-spacer distance isconfigured to improve the uniformity between the plate spacing at aclosed configuration.

The distance between neighboring spacers (e.g. the inter-spacerdistance) is 1 um or less, 5 um or less, 10 um or less, 20 um or less,30 um or less, 40 um or less, 50 um or less, 60 um or less, 70 um orless, 80 um or less, 90 um or less, 100 um or less, 200 um or less, 300um or less, 400 um or less, or a range between any two of the values.

In certain embodiments, the inter-spacer distance is at 400 or less, 500or less, 1 mm or less, 2 mm or less, 3 mm or less, 5 mm or less, 7 mm orless, 10 mm or less, or any range between the values. In certainembodiments, the inter-spacer distance is a 10 mm or less, 20 mm orless, 30 mm or less, 50 mm or less, 70 mm or less, 100 mm or less, orany range between the values.

The distance between neighboring spacers (e.g. the inter-spacerdistance) is selected so that for a given properties of the plates and asample, at the closed-configuration of the plates, the sample thicknessvariation between two neighboring spacers is, in certain embodiments, atmost 0.5%, 1%, 5%, 10%, 20%, 30%, 50%, 80%, or any range between thevalues; or in certain embodiments, at most 80%, 100%, 200%, 400%, or arange between any two of the values.

Clearly, for maintaining a given sample thickness variation between twoneighboring spacers, when a more flexible plate is used, a closerinter-spacer distance is needed.

Specify the accuracy of the inter spacer distance.

In a preferred embodiment, the spacer is a periodic square array,wherein the spacer is a pillar that has a height of 2 to 4 um, anaverage lateral dimension of from 5 to 20 um, and inter-spacer spacingof 1 um to 100 um.

In a preferred embodiment, the spacer is a periodic square array,wherein the spacer is a pillar that has a height of 2 to 4 um, anaverage lateral dimension of from 5 to 20 um, and inter-spacer spacingof 100 um to 250 um.

In a preferred embodiment, the spacer is a periodic square array,wherein the spacer is a pillar that has a height of 4 to 50 um, anaverage lateral dimension of from 5 to 20 um, and inter-spacer spacingof 1 um to 100 um.

In a preferred embodiment, the spacer is a periodic square array,wherein the spacer is a pillar that has a height of 4 to 50 um, anaverage lateral dimension of from 5 to 20 um, and inter-spacer spacingof 100 um to 250 um.

The period of spacer array is between 1 nm to 100 nm in one preferredembodiment, 100 nm to 500 nm in another preferred embodiment, 500 nm to1000 nm in a separate preferred embodiment, 1 um (e.g. 1000 nm) to 2 umin another preferred embodiment, 2 um to 3 um in a separate preferredembodiment, 3 um to 5 um in another preferred embodiment, 5 um to 10 umin a separate preferred embodiment, and 10 um to 50 um in anotherpreferred embodiment, 50 um to 100 um in a separate preferredembodiment, 100 um to 175 um in a separate preferred embodiment, and 175um to 300 um in a separate preferred embodiment.

(xi) Spacer Density. The spacers are arranged on the respective platesat a surface density of greater than one per um², greater than one per10 um², greater than one per 100 um², greater than one per 500 um²,greater than one per 1000 um², greater than one per 5000 um², greaterthan one per 0.01 mm², greater than one per 0.1 mm², greater than oneper 1 mm², greater than one per 5 mm², greater than one per 10 mm²,greater than one per 100 mm², greater than one per 1000 mm², greaterthan one per 10000 mm², or a range between any two of the values.

(3) the spacers are configured to not take significant surface area(volume) in a given sample area (volume);

(xii) Ratio of Spacer Volume to Sample Volume. In many embodiments, theratio of the spacer volume (e.g. the volume of the spacer) to samplevolume (e.g. the volume of the sample), and/or the ratio of the volumeof the spacers that are inside of the relevant volume of the sample tothe relevant volume of the sample are controlled for achieving certainadvantages. The advantages include, but not limited to, the uniformityof the sample thickness control, the uniformity of analytes, the sampleflow properties (e.g. flow speed, flow direction, etc.).

In certain embodiments, the ratio of the spacer volume r) to samplevolume, and/or the ratio of the volume of the spacers that are inside ofthe relevant volume of the sample to the relevant volume of the sampleis less than 100%, at most 99%, at most 70%, at most 50%, at most 30%,at most 10%, at most 5%, at most 3% at most 1%, at most 0.1%, at most0.01%, at most 0.001%, or a range between any of the values.

(xiii) Spacers Fixed to Plates. The inter spacer distance and theorientation of the spacers, which play a key role in the presentinvention, are preferably maintained during the process of bringing theplates from an open configuration to the closed configuration, and/orare preferably predetermined before the process from an openconfiguration to a closed configuration.

In certain embodiments of the present disclosure, spacers are fixed onone of the plates before bring the plates to the closed configuration.The term “a spacer is fixed with its respective plate” means that thespacer is attached to a plate and the attachment is maintained during ause of the plate. An example of “a spacer is fixed with its respectiveplate” is that a spacer is monolithically made of one piece of materialof the plate, and the position of the spacer relative to the platesurface does not change. An example of “a spacer is not fixed with itsrespective plate” is that a spacer is glued to a plate by an adhesive,but during a use of the plate, the adhesive cannot hold the spacer atits original location on the plate surface (e.g. the spacer moves awayfrom its original position on the plate surface).

In certain embodiments, at least one of the spacers are fixed to itsrespective plate. In certain embodiments, at two spacers are fixed toits respective plates. In certain embodiments, a majority of the spacersare fixed with their respective plates. In certain embodiments, all ofthe spacers are fixed with their respective plates.

In certain embodiments, a spacer is fixed to a plate monolithically.

In certain embodiments, the spacers are fixed to its respective plate byone or any combination of the following methods and/or configurations:attached to, bonded to, fused to, imprinted, and etched.

The term “imprinted” means that a spacer and a plate are fixedmonolithically by imprinting (e.g. embossing) a piece of a material toform the spacer on the plate surface. The material can be single layerof a material or multiple layers of the material.

The term “etched” means that a spacer and a plate are fixedmonolithically by etching a piece of a material to form the spacer onthe plate surface. The material can be single layer of a material ormultiple layers of the material.

The term “fused to” means that a spacer and a plate are fixedmonolithically by attaching a spacer and a plate together, the originalmaterials for the spacer and the plate fused into each other, and thereis clear material boundary between the two materials after the fusion.

The term “bonded to” means that a spacer and a plate are fixedmonolithically by binding a spacer and a plate by adhesion.

The term “attached to” means that a spacer and a plate are connectedtogether.

In certain embodiments, the spacers and the plate are made in the samematerials. In other embodiment, the spacers and the plate are made fromdifferent materials. In other embodiment, the spacer and the plate areformed in one piece. In other embodiment, the spacer has one end fixedto its respective plate, while the end is open for accommodatingdifferent configurations of the two plates.

In other embodiment, each of the spacers independently is at least oneof attached to, bonded to, fused to, imprinted in, and etched in therespective plate. The term “independently” means that one spacer isfixed with its respective plate by a same or a different method that isselected from the methods of attached to, bonded to, fused to, imprintedin, and etched in the respective plate.

In certain embodiments, at least a distance between two spacers ispredetermined (“predetermined inter-spacer distance” means that thedistance is known when a user uses the plates.).

In certain embodiments of all methods and devices described herein,there are additional spacers besides to the fixed spacers.

(xiv) Specific Sample Thickness. In present invention, it was observedthat a larger plate holding force (e.g. the force that holds the twoplates together) can be achieved by using a smaller plate spacing (for agiven sample area), or a larger sample area (for a given plate-spacing),or both.

In certain embodiments, at least one of the plates is transparent in aregion encompassing the relevant area, each plate has an inner surfaceconfigured to contact the sample in the closed configuration; the innersurfaces of the plates are substantially parallel with each other, inthe closed configuration; the inner surfaces of the plates aresubstantially planar, except the locations that have the spacers; or anycombination of thereof.

The spacers can be fabricated on a plate in a variety of ways, usinglithography, etching, embossing (nanoimprint), depositions, lift-off,fusing, or a combination of thereof. In certain embodiments, the spacersare directly embossed or imprinted on the plates. In certainembodiments, the spacers imprinted into a material (e.g. plastics) thatis deposited on the plates. In certain embodiments, the spacers are madeby directly embossing a surface of a CROF plate. The nanoimprinting canbe done by roll to roll technology using a roller imprinter, or roll toa planar nanoimprint. Such process has a great economic advantage andhence lowering the cost.

In certain embodiments, the spacers are deposited on the plates. Thedeposition can be evaporation, pasting, or a lift-off. In the pasting,the spacer is fabricated first on a carrier, then the spacer istransferred from the carrier to the plate. In the lift-off, a removablematerial is first deposited on the plate and holes are created in thematerial; the hole bottom expose the plate surface and then a spacermaterial is deposited into the hole and afterwards the removablematerial is removed, leaving only the spacers on the plate surface. Incertain embodiments, the spacers deposited on the plate are fused withthe plate. In certain embodiments, the spacer and the plates arefabricated in a single process. The single process includes imprinting(e.g. embossing, molding) or synthesis.

In certain embodiments, at least two of the spacers are fixed to therespective plate by different fabrication methods, and optionallywherein the different fabrication methods include at least one of beingdeposition, bonded, fuse, imprinted, and etched.

In certain embodiments, one or more of the spacers are fixed to therespective plate(s) is by a fabrication method of being bonded, beingfused, being imprinted, or being etched, or any combination of thereof.

In certain embodiments, the fabrication methods for forming suchmonolithic spacers on the plate include a method of being bonded, beingfused, being imprinted, or being etched, or any combination of thereof.

B) Adaptor

Details of the Adaptor are described in detail in a variety ofpublications including International Application No. PCT/US2018/017504.

The present invention that is described herein address this problem byproviding a system comprising an optical adaptor and a smartphone. Theoptical adaptor device fits over a smartphone converting it into amicroscope which can take both fluorescent and bright-field images of asample. This system can be operated conveniently and reliably by acommon person at any location. The optical adaptor takes advantage ofthe existing resources of the smartphone, including camera, lightsource, processor and display screen, which provides a low-cost solutionlet the user to do bright-field and fluorescent microscopy.

In this invention, the optical adaptor device comprises a holder framefitting over the upper part of the smartphone and an optical boxattached to the holder having sample receptacle slot and illuminationoptics. In some references (U.S. Pat. No. 2016/029091 and U.S. Pat. No.2011/0292198), their optical adaptor design is a whole piece includingboth the clip-on mechanics parts to fit over the smartphone and thefunctional optics elements. This design has the problem that they needto redesign the whole-piece optical adaptor for each specific model ofsmartphone. But in this present invention, the optical adaptor isseparated into a holder frame only for fitting a smartphone and auniversal optical box containing all the functional parts. For thesmartphones with different dimensions, as long as the relative positionsof the camera and the light source are the same, only the holder frameneed to be redesigned, which will save a lot of cost of design andmanufacture.

The optical box of the optical adaptor comprises: a receptacle slotwhich receives and position the sample in a sample slide in the field ofview and focal range of the smartphone camera; a bright-fieldillumination optics for capturing bright-field microscopy images of asample; a fluorescent illumination optics for capturing fluorescentmicroscopy images of a sample; a lever to switch between bright-fieldillumination optics and fluorescent illumination optics by slidinginward and outward in the optical box.

The receptacle slot has a rubber door attached to it, which can fullycover the slot to prevent the ambient light getting into the optical boxto be collected by the camera. In U.S. Pat. 2016/0290916, the sampleslot is always exposed to the ambient light which won't cause too muchproblem because it only does bright-field microscopy. But the presentinvention can take the advantage of this rubber door when doingfluorescent microscopy because the ambient light would bring a lot ofnoise to the image sensor of the camera.

To capture good fluorescent microscopy image, it is desirable thatnearly no excitation light goes into the camera and only the fluorescentemitted by the sample is collected by the camera. For all commonsmartphones, however, the optical filter putting in front of the cameracannot block the undesired wavelength range of the light emitted fromthe light source of a smartphone very well due to the large divergenceangle of the beams emitted by the light source and the optical filternot working well for un-collimated beams. Collimation optics can bedesigned to collimated the beam emitted by the smartphone light sourceto address this issue, but this approach increase the size and cost ofthe adaptor. Instead, in this present invention, fluorescentillumination optics enables the excitation light to illuminate thesample partially from the waveguide inside the sample slide andpartially from the backside of the sample side in large obliqueincidence angle so that excitation light will nearly not be collected bythe camera to reduce the noise signal getting into the camera.

The bright-field illumination optics in the adaptor receive and turn thebeam emitted by the light source so as to back-illuminated the sample innormal incidence angle.

Typically, the optical box also comprises a lens mounted in it alignedwith the camera of the smartphone, which magnifies the images capturedby the camera. The images captured by the camera can be furtherprocessed by the processor of smartphone and outputs the analysis resulton the screen of smartphone.

To achieve both bright-field illumination and fluorescent illuminationoptics in a same optical adaptor, in this present invention, a slidablelever is used. The optical elements of the fluorescent illuminationoptics are mounted on the lever and when the lever fully slides into theoptical box, the fluorescent illumination optics elements block theoptical path of bright-field illumination optics and switch theillumination optics to fluorescent illumination optics. And when thelever slides out, the fluorescent illumination optics elements mountedon the lever move out of the optical path and switch the illuminationoptics to bright-field illumination optics. This lever design makes theoptical adaptor work in both bright-field and fluorescent illuminationmodes without the need for designing two different single-mode opticalboxes.

The lever comprises two planes at different planes at different heights.

In certain embodiments, two planes can be joined together with avertical bar and move together in or out of the optical box. In certainembodiments, two planes can be separated and each plane can moveindividually in or out of the optical box.

The upper lever plane comprises at least one optical element which canbe, but not limited to be an optical filter. The upper lever plane movesunder the light source and the preferred distance between the upperlever plane and the light source is in the range of 0 to 5 mm.

Part of the bottom lever plane is not parallel to the image plane. Andthe surface of the non-parallel part of the bottom lever plane hasmirror finish with high reflectivity larger than 95%. The non-parallelpart of the bottom lever plane moves under the light source and deflectsthe light emitted from the light source to back-illuminate the samplearea right under the camera. The preferred tilt angle of thenon-parallel part of the bottom lever plane is in the range of 45 degreeto 65 degree and the tilt angle is defined as the angle between thenon-parallel bottom plane and the vertical plane.

Part of the bottom lever plane is parallel to the image plane and islocated under and 1 mm to 10 mm away from the sample. The surface of theparallel part of the bottom lever plane is highly light absorptive withlight absorption larger than 95%. This absorptive surface is toeliminate the reflective light back-illuminating on the sample in smallincidence angle.

To slide in and out to switch the illumination optics using the lever, astopper design comprising a ball plunger and a groove on the lever isused in order to stop the lever at a pre-defined position when beingpulled outward from the adaptor. This allow the user to use arbitraryforce the pull the lever but make the lever to stop at a fixed positionwhere the optical adaptor's working mode is switched to bright-filedillumination.

A sample slider is mounted inside the receptacle slot to receive theQMAX device and position the sample in the QMAX device in the field ofview and focal range of the smartphone camera.

The sample slider comprises a fixed track frame and a moveable arm:

The frame track is fixedly mounted in the receptacle slot of the opticalbox. And the track frame has a sliding track slot that fits the widthand thickness of the QMAX device so that the QMAX device can slide alongthe track. The width and height of the track slot is carefullyconfigured to make the QMAX device shift less than 0.5 mm in thedirection perpendicular to the sliding direction in the sliding planeand shift less than less than 0.2 mm along the thickness direction ofthe QMAX device.

The frame track has an opened window under the field of view of thecamera of smartphone to allow the light back-illuminate the sample.

A moveable arm is pre-built in the sliding track slot of the track frameand moves together with the QMAX device to guide the movement of QMAXdevice in the track frame.

The moveable arm equipped with a stopping mechanism with two pre-definedstop positions. For one position, the arm will make the QMAX device stopat the position where a fixed sample area on the QMAX device is rightunder the camera of smartphone. For the other position, the arm willmake the QMAX device stop at the position where the sample area on QMAXdevice is out of the field of view of the smartphone and the QMAX devicecan be easily taken out of the track slot.

The moveable arm switches between the two stop positions by a pressingthe QMAX device and the moveable arm together to the end of the trackslot and then releasing.

The moveable arm can indicate if the QMAX device is inserted in correctdirection. The shape of one corner of the QMAX device is configured tobe different from the other three right angle corners. And the shape ofthe moveable arm matches the shape of the corner with the special shapeso that only in correct direction can QMAX device slide to correctposition in the track slot.

C) Smartphone/Detection System

Details of the Smartphone/Detection System are described in detail in avariety of publications including International Application (IA) No.PCT/US2016/046437 filed on Aug. 10, 2016, IA No. PCT/US2016/051775 filedSep. 14, 2016, U.S. Provisional Application No. 62/456,065, which wasfiled on Feb. 7, 2017, U.S. Provisional Application Nos. 62/456,287 and62/456,590, which were filed on Feb. 8, 2017, U.S. ProvisionalApplication No. 62/456,504, which was filed on Feb. 8, 2017, U.S.Provisional Application No. 62/459,544, which was filed on Feb. 15,2017, and U.S. Provisional Application Nos. 62/460,075 and 62/459,920,which were filed on Feb. 16, 2017.

The devices/apparatus, systems, and methods herein disclosed can includeor use Q-cards for sample detection, analysis, and quantification. Incertain embodiments, the Q-card is used together with an adaptor thatcan connect the Q-card with a smartphone detection system. In certainembodiments, the smartphone comprises a camera and/or an illuminationsource. In certain embodiments, the smartphone comprises a camera, whichcan be used to capture images or the sample when the sample ispositioned in the field of view of the camera (e.g. by an adaptor). Incertain embodiments, the camera includes one set of lenses (e.g. as iniPhone™ 6). In certain embodiments, the camera includes at least twosets of lenses (e.g. as in iPhone™ 7). In certain embodiments, thesmartphone comprises a camera, but the camera is not used for imagecapturing.

In certain embodiments, the smartphone comprises a light source such asbut not limited to LED (light emitting diode). In certain embodiments,the light source is used to provide illumination to the sample when thesample is positioned in the field of view of the camera (e.g. by anadaptor). In certain embodiments, the light from the light source isenhanced, magnified, altered, and/or optimized by optical components ofthe adaptor.

In certain embodiments, the smartphone comprises a processor that isconfigured to process the information from the sample. The smartphoneincludes software instructions that, when executed by the processor, canenhance, magnify, and/or optimize the signals (e.g. images) from thesample. The processor can include one or more hardware components, suchas a central processing unit (CPU), an application-specific integratedcircuit (ASIC), an application-specific instruction-set processor(ASIP), a graphics processing unit (GPU), a physics processing unit(PPU), a digital signal processor (DSP), a field-programmable gate array(FPGA), a programmable logic device (PLD), a controller, amicrocontroller unit, a reduced instruction-set computer (RISC), amicroprocessor, or the like, or any combination thereof.

In certain embodiments, the smartphone comprises a communication unit,which is configured and/or used to transmit data and/or images relatedto the sample to another device. Merely by way of example, thecommunication unit can use a cable network, a wireline network, anoptical fiber network, a telecommunications network, an intranet, theInternet, a local area network (LAN), a wide area network (WAN), awireless local area network (WLAN), a metropolitan area network (MAN), awide area network (WAN), a public telephone switched network (PSTN), aBluetooth network, a ZigBee network, a near field communication (NFC)network, or the like, or any combination thereof. In certainembodiments, the smartphone is an iPhone™, an Android™ phone, or aWindows™ phone.

D). Method of Manufacture

Details of the Method of Manufacture are described in detail in avariety of publications including International Application No.PCT/US2018/057873 filed Oct. 26, 2018, which is hereby incorporated byreference herein for all purposes.

Devices of the disclosure can be fabricated using techniques well knownin the art. The choice of fabrication technique will depend on thematerial used for the device and the size of the spacer array and/or thesize of the spacers. Exemplary materials for fabricating the devices ofthe invention include glass, silicon, steel, nickel, polymers, e.g.,poly(methylmethacrylate) (PMMA), polycarbonate, polystyrene,polyethylene, polyolefins, silicones (e.g., poly(dimethylsiloxane)),polypropylene, cis-polyisoprene (rubber), poly(vinyl chloride) (PVC),poly(vinyl acetate) (PVAc), polychloroprene (neoprene),polytetrafluoroethylene (Teflon), poly(vinylidene chloride) (SaranA),and cyclic olefin polymer (COP) and cyclic olefin copolymer (COC), andcombinations thereof. Other materials are known in the art. For example,deep Reactive Ion Etch (DRIE) is used to fabricate silicon-based deviceswith small gaps, small spacers and large aspect ratios (ratio of spacerheight to lateral dimension). Thermoforming (embossing, injectionmolding) of plastic devices can also be used, e.g., when the smallestlateral feature is >20 microns and the aspect ratio of these features is10.

Additional methods include photolithography (e.g., stereolithography orx-ray photolithography), molding, embossing, silicon micromachining, wetor dry chemical etching, milling, diamond cutting, LithographicGalvanoformung and Abformung (LIGA), and electroplating. For example,for glass, traditional silicon fabrication techniques ofphotolithography followed by wet (KOH) or dry etching (reactive ionetching with fluorine or other reactive gas) can be employed. Techniquessuch as laser micromachining can be adopted for plastic materials withhigh photon absorption efficiency. This technique is suitable for lowerthroughput fabrication because of the serial nature of the process. Formass-produced plastic devices, thermoplastic injection molding, andcompression molding can be suitable. Conventional thermoplasticinjection molding used for mass-fabrication of compact discs (whichpreserves fidelity of features in sub-microns) can also be employed tofabricate the devices of the invention. For example, the device featuresare replicated on a glass master by conventional photolithography. Theglass master is electroformed to yield a tough, thermal shock resistant,thermally conductive, hard mold. This mold serves as the master templatefor injection molding or compression molding the features into a plasticdevice. Depending on the plastic material used to fabricate the devicesand the requirements on optical quality and throughput of the finishedproduct, compression molding or injection molding can be chosen as themethod of manufacture. Compression molding (also called hot embossing orrelief imprinting) has the advantages of being compatible with highmolecular weight polymers, which are excellent for small structures andcan replicate high aspect ratio structures but has longer cycle times.Injection molding works well for low aspect ratio structures and is mostsuitable for low molecular weight polymers.

A device can be fabricated in one or more pieces that are thenassembled. Layers of a device can be bonded together by clamps,adhesives, heat, anodic bonding, or reactions between surface groups(e.g., wafer bonding). Alternatively, a device with channels or gaps inmore than one plane can be fabricated as a single piece, e.g., usingstereolithography or other three-dimensional fabrication techniques.

To reduce non-specific adsorption of cells or compounds released bylysed cells onto the surfaces of the device, one or more surfaces of thedevice can be chemically modified to be non-adherent or repulsive. Thesurfaces can be coated with a thin film coating (e.g., a monolayer) ofcommercial non-stick reagents, such as those used to form hydrogels.Additional examples chemical species that can be used to modify thesurfaces of the device include oligoethylene glycols, fluorinatedpolymers, organosilanes, thiols, poly-ethylene glycol, hyaluronic acid,bovine serum albumin, poly-vinyl alcohol, mucin, poly-HEMA,methacrylated PEG, and agarose. Charged polymers can also be employed torepel oppositely charged species. The type of chemical species used forrepulsion and the method of attachment to the surfaces of the devicewill depend on the nature of the species being repelled and the natureof the surfaces and the species being attached. Such surfacemodification techniques are well known in the art. The surfaces can befunctionalized before or after the device is assembled. The surfaces ofthe device can also be coated in order to capture materials in thesample, e.g., membrane fragments or proteins.

In certain embodiments of the present disclosure, a method forfabricating any Q-Card of the present disclosure can comprise injectionmolding of the first plate. In certain embodiments of the presentdisclosure, a method for fabricating any Q-Card of the presentdisclosure can comprise nanoimprinting or extrusion printing of thesecond plate. In certain embodiments of the present disclosure, a methodfor fabricating any Q-Card of the present disclosure can comprise Lasercutting the first plate. In certain embodiments of the presentdisclosure, a method for fabricating any Q-Card of the presentdisclosure can comprise nanoimprinting or extrusion printing of thesecond plate. In certain embodiments of the present disclosure, a methodfor fabricating any Q-Card of the present disclosure can compriseinjection molding and laser cutting the first plate. In certainembodiments of the present disclosure, a method for fabricating anyQ-Card of the present disclosure can comprise nanoimprinting orextrusion printing of the second plate. In certain embodiments of thepresent disclosure, a method for fabricating any Q-Card of the presentdisclosure can comprise nanoimprinting or extrusion printing tofabricated both the first and the second plate. In certain embodimentsof the present disclosure, a method for fabricating any Q-Card of thepresent disclosure can comprise fabricating the first plate or thesecond plate, using injection molding, laser cutting the first plate,nanoimprinting, extrusion printing, or a combination of thereof. Incertain embodiments of the present disclosure, a method for fabricatingany Q-Card of the present disclosure can comprise a step of attachingthe hinge on the first and the second plates after the fabrication ofthe first and second plates.

E) Sample Types & Subjects

Details of the Samples & Subjects are described in detail in a varietyof publications including International Application (IA) No.PCT/US2016/046437 filed on Aug. 10, 2016, IA No. PCT/US2016/051775 filedon Sep. 14, 2016, IA No. PCT/US201/017307 filed on Feb. 7, 2018, IA No.and PCT/US2017/065440 filed on Dec. 8, 2017.

A sample can be obtained from a subject. A subject as described hereincan be of any age and can be an adult, infant or child. In some cases,the subject is 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88,89, 90, 91, 92, 93, 94, 95, 96, 97, 98, or 99 years old, or within arange therein (e.g., between 2 and 20 years old, between 20 and 40 yearsold, or between 40 and 90 years old). A particular class of subjectsthat can benefit is subjects who have or are suspected of having aninfection (e.g., a bacterial and/or a viral infection). Anotherparticular class of subjects that can benefit is subjects who can be athigher risk of getting an infection. Furthermore, a subject treated byany of the methods or compositions described herein can be male orfemale. Any of the methods, devices, or kits disclosed herein can alsobe performed on a non-human subject, such as a laboratory or farmanimal. Non-limiting examples of a non-human subjects include a dog, agoat, a guinea pig, a hamster, a mouse, a pig, a non-human primate(e.g., a gorilla, an ape, an orangutan, a lemur, or a baboon), a rat, asheep, a cow, or a zebrafish.

The devices, apparatus, systems, and methods herein disclosed can beused for samples such as but not limited to diagnostic samples, clinicalsamples, environmental samples and foodstuff samples.

For example, in certain embodiments, the devices, apparatus, systems,and methods herein disclosed are used for a sample that includes cells,tissues, bodily fluids and/or a mixture thereof. In certain embodiments,the sample comprises a human body fluid. In certain embodiments, thesample comprises at least one of cells, tissues, bodily fluids, stool,amniotic fluid, aqueous humour, vitreous humour, blood, whole blood,fractionated blood, plasma, serum, breast milk, cerebrospinal fluid,cerumen, chyle, chime, endolymph, perilymph, feces, gastric acid,gastric juice, lymph, mucus, nasal drainage, phlegm, pericardial fluid,peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum, semen,sputum, sweat, synovial fluid, tears, vomit, urine, and exhaled breathcondensate.

In certain embodiments, the devices, apparatus, systems, and methodsherein disclosed are used for an environmental sample that is obtainedfrom any suitable source, such as but not limited to: river, lake, pond,ocean, glaciers, icebergs, rain, snow, sewage, reservoirs, tap water,drinking water, etc.; solid samples from soil, compost, sand, rocks,concrete, wood, brick, sewage, etc.; and gaseous samples from the air,underwater heat vents, industrial exhaust, vehicular exhaust, etc. Incertain embodiments, the environmental sample is fresh from the source;in certain embodiments, the environmental sample is processed. Forexample, samples that are not in liquid form are converted to liquidform before the subject devices, apparatus, systems, and methods areapplied.

In certain embodiments, the devices, apparatus, systems, and methodsherein disclosed are used for a foodstuff sample, which is suitable orhas the potential to become suitable for animal consumption, e.g., humanconsumption. In certain embodiments, a foodstuff sample includes rawingredients, cooked or processed food, plant and animal sources of food,preprocessed food as well as partially or fully processed food, etc. Incertain embodiments, samples that are not in liquid form are convertedto liquid form before the subject devices, apparatus, systems, andmethods are applied.

The subject devices, apparatus, systems, and methods can be used toanalyze any volume of the sample. Examples of the volumes include, butare not limited to, about 10 mL or less, 5 mL or less, 3 mL or less, 1microliter (uL, also “uL” herein) or less, 500 uL or less, 300 uL orless, 250 uL or less, 200 uL or less, 170 uL or less, 150 uL or less,125 uL or less, 100 uL or less, 75 uL or less, 50 uL or less, 25 uL orless, 20 uL or less, 15 uL or less, 10 uL or less, 5 uL or less, 3 uL orless, 1 uL or less, 0.5 uL or less, 0.1 uL or less, 0.05 uL or less,0.001 uL or less, 0.0005 uL or less, 0.0001 uL or less, 10 pL or less, 1pL or less, or a range between any two of the values.

In certain embodiments, the volume of the sample includes, but is notlimited to, about 100 uL or less, 75 uL or less, 50 uL or less, 25 uL orless, 20 uL or less, 15 uL or less, 10 uL or less, 5 uL or less, 3 uL orless, 1 uL or less, 0.5 uL or less, 0.1 uL or less, 0.05 uL or less,0.001 uL or less, 0.0005 uL or less, 0.0001 uL or less, 10 pL or less, 1pL or less, or a range between any two of the values. In certainembodiments, the volume of the sample includes, but is not limited to,about 10 uL or less, 5 uL or less, 3 uL or less, 1 uL or less, 0.5 uL orless, 0.1 uL or less, 0.05 uL or less, 0.001 uL or less, 0.0005 uL orless, 0.0001 uL or less, 10 pL or less, 1 pL or less, or a range betweenany two of the values.

In certain embodiments, the amount of the sample is about a drop ofliquid. In certain embodiments, the amount of sample is the amountcollected from a pricked finger or fingerstick. In certain embodiments,the amount of sample is the amount collected from a microneedle,micropipette or a venous draw.

F) Machine Learning

Details of the Network are described in detail in a variety ofpublications including International Application (IA) No.PCT/US2018/017504 filed Feb. 8, 2018, and PCT/US2018/057877 filed Oct.26, 2018, each of which are hereby incorporated by reference herein forall purposes.

One aspect of the present invention provides a framework of machinelearning and deep learning for analyte detection and localization. Amachine learning algorithm is an algorithm that is able to learn fromdata. A more rigorous definition of machine learning is “A computerprogram is said to learn from experience E with respect to some class oftasks T and performance measure P, if its performance at tasks in T, asmeasured by P, improves with experience E.” It explores the study andconstruction of algorithms that can learn from and make predictions ondata—such algorithms overcome the static program instructions by makingdata driven predictions or decisions, through building a model fromsample inputs.

Deep learning is a specific kind of machine learning based on a set ofalgorithms that attempt to model high level abstractions in data. In asimple case, there might be two sets of neurons: ones that receive aninput signal and ones that send an output signal. When the input layerreceives an input, it passes on a modified version of the input to thenext layer. In a deep network, there are many layers between the inputand output (and the layers are not made of neurons but it can help tothink of it that way), allowing the algorithm to use multiple processinglayers, composed of multiple linear and non-linear transformations.

One aspect of the present invention is to provide two analyte detectionand localization approaches. The first approach is a deep learningapproach and the second approach is a combination of deep learning andcomputer vision approaches.

(i) Deep Learning Approach. In the first approach, the disclosed analytedetection and localization workflow consists of two stages, training andprediction. We describe training and prediction stages in the followingparagraphs.

(a) Training Stage

In the training stage, training data with annotation is fed into aconvolutional neural network. Convolutional neural network is aspecialized neural network for processing data that has a grid-like,feed forward and layered network topology. Examples of the data includetime-series data, which can be thought of as a 1D grid taking samples atregular time intervals, and image data, which can be thought of as a 2Dgrid of pixels. Convolutional networks have been successful in practicalapplications. The name “convolutional neural network” indicates that thenetwork employs a mathematical operation called convolution. Convolutionis a specialized kind of linear operation. Convolutional networks aresimply neural networks that use convolution in place of general matrixmultiplication in at least one of their layers.

The machine learning model receives one or multiple images of samplesthat contain the analytes taken by the imaging system over the sampleholding QMAX device as training data. Training data are annotated foranalytes to be assayed, wherein the annotations indicate whether or notanalytes are in the training data and where they locate in the image.Annotation can be done in the form of tight bounding boxes which fullycontains the analyte, or center locations of analytes. In the lattercase, center locations are further converted into circles coveringanalytes or a Gaussian kernel in a point map.

When the size of training data is large, training machine learning modelpresents two challenges: annotation (usually done by human) is timeconsuming, and the training is computationally expensive. To overcomethese challenges, one can partition the training data into patches ofsmall size, then annotate and train on these patches, or a portion ofthese patches. The term “machine learning” can refer to algorithms,systems and apparatus in the field of artificial intelligence that oftenuse statistical techniques and artificial neural network trained fromdata without being explicitly programmed

The annotated images are fed to the machine learning (ML) trainingmodule, and the model trainer in the machine learning module will traina ML model from the training data (annotated sample images). The inputdata will be fed to the model trainer in multiple iterations untilcertain stopping criterion is satisfied. The output of the ML trainingmodule is a ML model—a computational model that is built from a trainingprocess in the machine learning from the data that gives computer thecapability to perform certain tasks (e.g. detect and classify theobjects) on its own.

The trained machine learning model is applied during the predication (orinference) stage by the computer. Examples of machine learning modelsinclude ResNet, DenseNet, etc. which are also named as “deep learningmodels” because of the depth of the connected layers in their networkstructure. In certain embodiments, the Caffe library with fullyconvolutional network (FCN) was used for model training and predication,and other convolutional neural network architecture and library can alsobe used, such as TensorFlow.

The training stage generates a model that will be used in the predictionstage. The model can be repeatedly used in the prediction stage forassaying the input. Thus, the computing unit only needs access to thegenerated model. It does not need access to the training data, norrequiring the training stage to be run again on the computing unit.

(b) Prediction Stage

In the predication/inference stage, a detection component is applied tothe input image, and an input image is fed into the predication(inference) module preloaded with a trained model generated from thetraining stage. The output of the prediction stage can be bounding boxesthat contain the detected analytes with their center locations or apoint map indicating the location of each analyte, or a heatmap thatcontains the information of the detected analytes.

When the output of the prediction stage is a list of bounding boxes, thenumber of analytes in the image of the sample for assaying ischaracterized by the number of detected bounding boxes. When the outputof the prediction stage is a point map, the number of analytes in theimage of the sample for assaying is characterized by the integration ofthe point map. When the output of the prediction is a heatmap, alocalization component is used to identify the location and the numberof detected analytes is characterized by the entries of the heatmap.

One embodiment of the localization algorithm is to sort the heatmapvalues into a one-dimensional ordered list, from the highest value tothe lowest value. Then pick the pixel with the highest value, remove thepixel from the list, along with its neighbors. Iterate the process topick the pixel with the highest value in the list, until all pixels areremoved from the list.

In the detection component using heatmap, an input image, along with themodel generated from the training stage, is fed into a convolutionalneural network, and the output of the detection stage is a pixel-levelprediction, in the form of a heatmap. The heatmap can have the same sizeas the input image, or it can be a scaled down version of the inputimage, and it is the input to the localization component. We disclose analgorithm to localize the analyte center. The main idea is toiteratively detect local peaks from the heatmap. After the peak islocalized, we calculate the local area surrounding the peak but withsmaller value. We remove this region from the heatmap and find the nextpeak from the remaining pixels. The process is repeated only all pixelsare removed from the heatmap.

In certain embodiments, the present invention provides the localizationalgorithm to sort the heatmap values into a one-dimensional orderedlist, from the highest value to the lowest value. Then pick the pixelwith the highest value, remove the pixel from the list, along with itsneighbors. Iterate the process to pick the pixel with the highest valuein the list, until all pixels are removed from the list.

Algorithm Global Search (heatmap) Input:   heatmap Output:   loci loci←{ } sort(heatmap) while (heatmap is not empty) {  s ← pop(heatmap)  D ←{disk center as s with radius R}  heatmap = heatmap \ D // remove D fromthe heatmap  add s to loci }

After sorting, heatmap is a one-dimensional ordered list, where theheatmap value is ordered from the highest to the lowest. Each heatmapvalue is associated with its corresponding pixel coordinates. The firstitem in the heatmap is the one with the highest value, which is theoutput of the pop(heatmap) function. One disk is created, where thecenter is the pixel coordinate of the one with highest heatmap value.Then all heatmap values whose pixel coordinates resides inside the diskis removed from the heatmap. The algorithm repeatedly pops up thehighest value in the current heatmap, removes the disk around it, tillthe items are removed from the heatmap.

In the ordered list heatmap, each item has the knowledge of theproceeding item, and the following item. When removing an item from theordered list, we make the following changes:

-   -   Assume the removing item is x_(r), its proceeding item is x_(p),        and its following item is x_(f).    -   For the proceeding item x_(p), re-define its following item to        the following item of the removing item. Thus, the following        item of x_(p) is now x_(f).    -   For the removing item x_(r), un-define its proceeding item and        following item, which removes it from the ordered list.    -   For the following item x_(f), re-define its proceeding item to        the proceeding item of the removed item. Thus, the proceeding        item of x_(f) is now x_(p).

After all items are removed from the ordered list, the localizationalgorithm is complete. The number of elements in the set loci will bethe count of analytes, and location information is the pixel coordinatefor each s in the set loci.

Another embodiment searches local peak, which is not necessary the onewith the highest heatmap value. To detect each local peak, we start froma random starting point, and search for the local maximal value. Afterwe find the peak, we calculate the local area surrounding the peak butwith smaller value. We remove this region from the heatmap and find thenext peak from the remaining pixels. The process is repeated only allpixels are removed from the heatmap.

Algorithm LocalSearch (s, heatmap) Input:  s: starting location (x, y) heatmap Output:  s: location of local peak. We only consider pixels ofvalue > 0. Algorithm Cover (s, heatmap) Input:  s: location of localpeak.  heatmap: Output:  cover: a set of pixels covered by peak:

This is a breadth-first-search algorithm starting from s, with onealtered condition of visiting points: a neighbor p of the currentlocation q is only added to cover if heatmap[p]>0 andheatmap[p]<=heatmap[q]. Therefore, each pixel in cover has anon-descending path leading to the local peak s.

Algorithm Localization (heatmap) Input:   heatmap Output:   loci loci ←{} pixels ←{all pixels from heatmap} while pixels is not empty {  s ←anypixel from pixels  s ←LocalSearch(s, heatmap) // s is now local peak probe local region of radius R surrounding s for better local peak  r←Cover(s, heatmap)  pixels ← pixels \ r    // remove all pixels in cover add s to loci

(ii) Mixture of Deep Learning and Computer Vision Approaches. In thesecond approach, the detection and localization are realized by computervision algorithms, and a classification is realized by deep learningalgorithms, wherein the computer vision algorithms detect and locatepossible candidates of analytes, and the deep learning algorithmclassifies each possible candidate as a true analyte and false analyte.The location of all true analyte (along with the total count of trueanalytes) will be recorded as the output.

(a) Detection. The computer vision algorithm detects possible candidatebased on the characteristics of analytes, including but not limited tointensity, color, size, shape, distribution, etc. A pre-processingscheme can improve the detection. Pre-processing schemes includecontrast enhancement, histogram adjustment, color enhancement,de-nosing, smoothing, de-focus, etc. After pre-processing, the inputimage is sent to a detector. The detector tells the existing of possiblecandidate of analyte and gives an estimate of its location. Thedetection can be based on the analyte structure (such as edge detection,line detection, circle detection, etc.), the connectivity (such as blobdetection, connect components, contour detection, etc.), intensity,color, shape using schemes such as adaptive thresholding, etc.

(b) Localization. After detection, the computer vision algorithm locateseach possible candidate of analytes by providing its boundary or a tightbounding box containing it. This can be achieved through objectsegmentation algorithms, such as adaptive thresholding, backgroundsubtraction, floodfill, mean shift, watershed, etc. Very often, thelocalization can be combined with detection to produce the detectionresults along with the location of each possible candidates of analytes.

(c) Classification. The deep learning algorithms, such as convolutionalneural networks, achieve start-of-the-art visual classification. Weemploy deep learning algorithms for classification on each possiblecandidate of analytes. Various convolutional neural network can beutilized for analyte classification, such as VGGNet, ResNet, MobileNet,DenseNet, etc.

Given each possible candidate of analyte, the deep learning algorithmcomputes through layers of neurons via convolution filters andnon-linear filters to extract high-level features that differentiateanalyte against non-analytes. A layer of fully convolutional networkwill combine high-level features into classification results, whichtells whether it is a true analyte or not, or the probability of being aanalyte.

G). Applications, Bio/Chemical Biomarkers, and Health Conditions

The applications of the present invention include, but not limited to,(a) the detection, purification and quantification of chemical compoundsor biomolecules that correlates with the stage of certain diseases,e.g., infectious and parasitic disease, injuries, cardiovasculardisease, cancer, mental disorders, neuropsychiatric disorders andorganic diseases, e.g., pulmonary diseases, renal diseases, (b) thedetection, purification and quantification of microorganism, e.g.,virus, fungus and bacteria from environment, e.g., water, soil, orbiological samples, e.g., tissues, bodily fluids, (c) the detection,quantification of chemical compounds or biological samples that posehazard to food safety or national security, e.g. toxic waste, anthrax,(d) quantification of vital parameters in medical or physiologicalmonitor, e.g., glucose, blood oxygen level, total blood count, (e) thedetection and quantification of specific DNA or RNA from biosamples,e.g., cells, viruses, bodily fluids, (f) the sequencing and comparing ofgenetic sequences in DNA in the chromosomes and mitochondria for genomeanalysis or (g) to detect reaction products, e.g., during synthesis orpurification of pharmaceuticals.

The detection can be carried out in various sample matrix, such ascells, tissues, bodily fluids, and stool. Bodily fluids of interestinclude but are not limited to, amniotic fluid, aqueous humour, vitreoushumour, blood (e.g., whole blood, fractionated blood, plasma, serum,etc.), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle,chime, endolymph, perilymph, feces, gastric acid, gastric juice, lymph,mucus (including nasal drainage and phlegm), pericardial fluid,peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum (skin oil),semen, sputum, sweat, synovial fluid, tears, vomit, urine and exhaledcondensate. In certain embodiments, the sample comprises a human bodyfluid. In certain embodiments, the sample comprises at least one ofcells, tissues, bodily fluids, stool, amniotic fluid, aqueous humour,vitreous humour, blood, whole blood, fractionated blood, plasma, serum,breast milk, cerebrospinal fluid, cerumen, chyle, chime, endolymph,perilymph, feces, gastric acid, gastric juice, lymph, mucus, nasaldrainage, phlegm, pericardial fluid, peritoneal fluid, pleural fluid,pus, rheum, saliva, sebum, semen, sputum, sweat, synovial fluid, tears,vomit, urine, and exhaled condensate.

In some embodiments, the sample is at least one of a biological sample,an environmental sample, and a biochemical sample.

The devices, systems and the methods in the present invention find usein a variety of different applications in various fields, wheredetermination of the presence or absence, and/or quantification of oneor more analytes in a sample are desired. For example, the subjectmethod finds use in the detection of proteins, peptides, nucleic acids,synthetic compounds, inorganic compounds, and the like. The variousfields include, but not limited to, human, veterinary, agriculture,foods, environments, drug testing, and others.

In certain embodiments, the subject method finds use in the detection ofnucleic acids, proteins, or other biomolecules in a sample. The methodscan include the detection of a set of biomarkers, e.g., two or moredistinct protein or nucleic acid biomarkers, in a sample. For example,the methods can be used in the rapid, clinical detection of two or moredisease biomarkers in a biological sample, e.g., as can be employed inthe diagnosis of a disease condition in a subject, or in the ongoingmanagement or treatment of a disease condition in a subject, etc. Asdescribed above, communication to a physician or other health-careprovider can better ensure that the physician or other health-careprovider is made aware of, and cognizant of, possible concerns and canthus be more likely to take appropriate action.

The applications of the devices, systems and methods in the presentinventions of employing a CROF device include, but are not limited to,(a) the detection, purification and quantification of chemical compoundsor biomolecules that correlates with the stage of certain diseases,e.g., infectious and parasitic disease, injuries, cardiovasculardisease, cancer, mental disorders, neuropsychiatric disorders andorganic diseases, e.g., pulmonary diseases, renal diseases, (b) thedetection, purification and quantification of microorganism, e.g.,virus, fungus and bacteria from environment, e.g., water, soil, orbiological samples, e.g., tissues, bodily fluids, (c) the detection,quantification of chemical compounds or biological samples that posehazard to food safety or national security, e.g. toxic waste, anthrax,(d) quantification of vital parameters in medical or physiologicalmonitor, e.g., glucose, blood oxygen level, total blood count, (e) thedetection and quantification of specific DNA or RNA from biosamples,e.g., cells, viruses, bodily fluids, (f) the sequencing and comparing ofgenetic sequences in DNA in the chromosomes and mitochondria for genomeanalysis or (g) to detect reaction products, e.g., during synthesis orpurification of pharmaceuticals. Some of the specific applications ofthe devices, systems and methods in the present invention are describednow in further detail.

The applications of the present invention include, but not limited to,(a) the detection, purification and quantification of chemical compoundsor biomolecules that correlates with the stage of certain diseases,e.g., infectious and parasitic disease, injuries, cardiovasculardisease, cancer, mental disorders, neuropsychiatric disorders andorganic diseases, e.g., pulmonary diseases, renal diseases, (b) thedetection, purification and quantification of microorganism, e.g.,virus, fungus and bacteria from environment, e.g., water, soil, orbiological samples, e.g., tissues, bodily fluids, (c) the detection,quantification of chemical compounds or biological samples that posehazard to food safety or national security, e.g. toxic waste, anthrax,(d) quantification of vital parameters in medical or physiologicalmonitor, e.g., glucose, blood oxygen level, total blood count, (e) thedetection and quantification of specific DNA or RNA from biosamples,e.g., cells, viruses, bodily fluids, (f) the sequencing and comparing ofgenetic sequences in DNA in the chromosomes and mitochondria for genomeanalysis or (g) to detect reaction products, e.g., during synthesis orpurification of pharmaceuticals.

An implementation of the devices, systems and methods in the presentinvention can include a) obtaining a sample, b) applying the sample toCROF device containing a capture agent that binds to an analyte ofinterest, under conditions suitable for binding of the analyte in asample to the capture agent, c) washing the CROF device, and d) readingthe CROF device, thereby obtaining a measurement of the amount of theanalyte in the sample. In certain embodiments, the analyte can be abiomarker, an environmental marker, or a foodstuff marker. The sample insome instances is a liquid sample, and can be a diagnostic sample (suchas saliva, serum, blood, sputum, urine, sweat, lacrima, semen, ormucus); an environmental sample obtained from a river, ocean, lake,rain, snow, sewage, sewage processing runoff, agricultural runoff,industrial runoff, tap water or drinking water; or a foodstuff sampleobtained from tap water, drinking water, prepared food, processed foodor raw food.

In any embodiment, the CROF device can be placed in a microfluidicdevice and the applying step b) can include applying a sample to amicrofluidic device comprising the CROF device.

In any embodiment, the reading step d) can include detecting afluorescence or luminescence signal from the CROF device.

In any embodiment, the reading step d) can include reading the CROFdevice with a handheld device configured to read the CROF device. Thehandheld device can be a mobile phone, e.g., a smart phone.

In any embodiment, the CROF device can include a labeling agent that canbind to an analyte-capture agent complex on the CROF device.

In any embodiment, the devices, systems and methods in the presentinvention can further include, between steps c) and d), the steps ofapplying to the CROF device a labeling agent that binds to ananalyte-capture agent complex on the CROF device, and washing the CROFdevice.

In any embodiment, the reading step d) can include reading an identifierfor the CROF device. The identifier can be an optical barcode, a radiofrequency ID tag, or combinations thereof.

In any embodiment, the devices, systems and methods in the presentinvention can further include applying a control sample to a controlCROF device containing a capture agent that binds to the analyte,wherein the control sample includes a known detectable amount of theanalyte, and reading the control CROF device, thereby obtaining acontrol measurement for the known detectable amount of the analyte in asample.

In any embodiment, the sample can be a diagnostic sample obtained from asubject, the analyte can be a biomarker, and the measured amount of theanalyte in the sample can be diagnostic of a disease or a condition.

In any embodiment, the devices, systems and methods in the presentinvention can further include receiving or providing to the subject areport that indicates the measured amount of the biomarker and a rangeof measured values for the biomarker in an individual free of or at lowrisk of having the disease or condition, wherein the measured amount ofthe biomarker relative to the range of measured values is diagnostic ofa disease or condition.

In any embodiment, the devices, systems and methods in the presentinvention can further include diagnosing the subject based oninformation including the measured amount of the biomarker in thesample. In some cases, the diagnosing step includes sending datacontaining the measured amount of the biomarker to a remote location andreceiving a diagnosis based on information including the measurementfrom the remote location.

In any embodiment, the applying step b) can include isolating miRNA fromthe sample to generate an isolated miRNA sample, and applying theisolated miRNA sample to the disk-coupled dots-on-pillar antenna (CROFdevice) array.

In any embodiment, the method can include receiving or providing areport that indicates the safety or harmfulness for a subject to beexposed to the environment from which the sample was obtained.

In any embodiment, the method can include sending data containing themeasured amount of the environmental marker to a remote location andreceiving a report that indicates the safety or harmfulness for asubject to be exposed to the environment from which the sample wasobtained.

In any embodiment, the CROF device array can include a plurality ofcapture agents that each binds to an environmental marker, and whereinthe reading step d) can include obtaining a measure of the amount of theplurality of environmental markers in the sample.

In any embodiment, the sample can be a foodstuff sample, wherein theanalyte can be a foodstuff marker, and wherein the amount of thefoodstuff marker in the sample can correlate with safety of thefoodstuff for consumption.

In any embodiment, the method can include receiving or providing areport that indicates the safety or harmfulness for a subject to consumethe foodstuff from which the sample is obtained.

In any embodiment, the method can include sending data containing themeasured amount of the foodstuff marker to a remote location andreceiving a report that indicates the safety or harmfulness for asubject to consume the foodstuff from which the sample is obtained.

In any embodiment, the CROF device array can include a plurality ofcapture agents that each binds to a foodstuff marker, wherein theobtaining can include obtaining a measure of the amount of the pluralityof foodstuff markers in the sample, and wherein the amount of theplurality of foodstuff marker in the sample can correlate with safety ofthe foodstuff for consumption.

Also provided herein are kits that find use in practicing the devices,systems and methods in the present invention.

The amount of sample can be about a drop of a sample. The amount ofsample can be the amount collected from a pricked finger or fingerstick.The amount of sample can be the amount collected from a microneedle or avenous draw.

A sample can be used without further processing after obtaining it fromthe source, or can be processed, e.g., to enrich for an analyte ofinterest, remove large particulate matter, dissolve or resuspend a solidsample, etc.

Any suitable method of applying a sample to the CROF device can beemployed. Suitable methods can include using a pipet, dropper, syringe,etc. In certain embodiments, when the CROF device is located on asupport in a dipstick format, as described below, the sample can beapplied to the CROF device by dipping a sample-receiving area of thedipstick into the sample.

A sample can be collected at one time, or at a plurality of times.Samples collected over time can be aggregated and/or processed (byapplying to a CROF device and obtaining a measurement of the amount ofanalyte in the sample, as described herein) individually. In someinstances, measurements obtained over time can be aggregated and can beuseful for longitudinal analysis over time to facilitate screening,diagnosis, treatment, and/or disease prevention.

Washing the CROF device to remove unbound sample components can be donein any convenient manner, as described above. In certain embodiments,the surface of the CROF device is washed using binding buffer to removeunbound sample components.

Detectable labeling of the analyte can be done by any convenient method.The analyte can be labeled directly or indirectly. In direct labeling,the analyte in the sample is labeled before the sample is applied to theCROF device. In indirect labeling, an unlabeled analyte in a sample islabeled after the sample is applied to the CROF device to capture theunlabeled analyte, as described below.

The samples from a subject, the health of a subject, and otherapplications of the present invention are further described below.Exemplary samples, health conditions, and application are also disclosedin, e.g., U.S. Pub. Nos. 2014/0154668 and 2014/0045209, which are herebyincorporated by reference.

The present inventions find use in a variety of applications, where suchapplications are generally analyte detection applications in which thepresence of a particular analyte in a given sample is detected at leastqualitatively, if not quantitatively. Protocols for carrying out analytedetection assays are well known to those of skill in the art and neednot be described in great detail here. Generally, the sample suspectedof comprising an analyte of interest is contacted with the surface of asubject nanosensor under conditions sufficient for the analyte to bindto its respective capture agent that is tethered to the sensor. Thecapture agent has highly specific affinity for the targeted molecules ofinterest. This affinity can be antigen-antibody reaction whereantibodies bind to specific epitope on the antigen, or a DNA/RNA orDNA/RNA hybridization reaction that is sequence-specific between two ormore complementary strands of nucleic acids. Thus, if the analyte ofinterest is present in the sample, it likely binds to the sensor at thesite of the capture agent and a complex is formed on the sensor surface.Namely, the captured analytes are immobilized at the sensor surface.After removing the unbounded analytes, the presence of this bindingcomplex on the surface of the sensor (e.g. the immobilized analytes ofinterest) is then detected, e.g., using a labeled secondary captureagent.

Specific analyte detection applications of interest includehybridization assays in which the nucleic acid capture agents areemployed and protein binding assays in which polypeptides, e.g.,antibodies, are employed. In these assays, a sample is first preparedand following sample preparation, the sample is contacted with a subjectnanosensor under specific binding conditions, whereby complexes areformed between target nucleic acids or polypeptides (or other molecules)that are complementary to capture agents attached to the sensor surface.

In one embodiment, the capture oligonucleotide is synthesized singlestrand DNA of 20-100 bases length, that is thiolated at one end. Thesemolecules are immobilized on the nanodevices' surface to capture thetargeted single-strand DNA (which can be at least 50 bp length) that hasa sequence that is complementary to the immobilized capture DNA. Afterthe hybridization reaction, a detection single strand DNA (which can beof 20-100 bp in length) whose sequence are complementary to the targetedDNA's unoccupied nucleic acid is added to hybridize with the target. Thedetection DNA has its one end conjugated to a fluorescence label, whoseemission wavelength are within the plasmonic resonance of thenanodevice. Therefore by detecting the fluorescence emission emanatefrom the nanodevices' surface, the targeted single strand DNA can beaccurately detected and quantified. The length for capture and detectionDNA determine the melting temperature (nucleotide strands will separateabove melting temperature), the extent of misparing (the longer thestrand, the lower the misparing).

One of the concerns of choosing the length for complementary bindingdepends on the needs to minimize misparing while keeping the meltingtemperature as high as possible. In addition, the total length of thehybridization length is determined in order to achieve optimum signalamplification.

A subject sensor can be employed in a method of diagnosing a disease orcondition, comprising: (a) obtaining a liquid sample from a patientsuspected of having the disease or condition, (b) contacting the samplewith a subject nanosensor, wherein the capture agent of the nanosensorspecifically binds to a biomarker for the disease and wherein thecontacting is done under conditions suitable for specific binding of thebiomarker with the capture agent; (c) removing any biomarker that is notbound to the capture agent; and (d) reading a light signal frombiomarker that remain bound to the nanosensor, wherein a light signalindicates that the patient has the disease or condition, wherein themethod further comprises labeling the biomarker with a light-emittinglabel, either prior to or after it is bound to the capture agent. Aswill be described in greater detail below, the patient can suspect ofhaving cancer and the antibody binds to a cancer biomarker. In otherembodiments, the patient is suspected of having a neurological disorderand the antibody binds to a biomarker for the neurological disorder.

The applications of the subject sensor include, but not limited to, (a)the detection, purification and quantification of chemical compounds orbiomolecules that correlates with the stage of certain diseases, e.g.,infectious and parasitic disease, injuries, cardiovascular disease,cancer, mental disorders, neuropsychiatric disorders and organicdiseases, e.g., pulmonary diseases, renal diseases, (b) the detection,purification and quantification of microorganism, e.g., virus, fungusand bacteria from environment, e.g., water, soil, or biological samples,e.g., tissues, bodily fluids, (c) the detection, quantification ofchemical compounds or biological samples that pose hazard to food safetyor national security, e.g. toxic waste, anthrax, (d) quantification ofvital parameters in medical or physiological monitor, e.g., glucose,blood oxygen level, total blood count, (e) the detection andquantification of specific DNA or RNA from biosamples, e.g., cells,viruses, bodily fluids, (f) the sequencing and comparing of geneticsequences in DNA in the chromosomes and mitochondria for genome analysisor (g) to detect reaction products, e.g., during synthesis orpurification of pharmaceuticals.

The detection can be carried out in various sample matrix, such ascells, tissues, bodily fluids, and stool. Bodily fluids of interestinclude but are not limited to, amniotic fluid, aqueous humour, vitreoushumour, blood (e.g., whole blood, fractionated blood, plasma, serum,etc.), breast milk, cerebrospinal fluid (CSF), cerumen (earwax), chyle,chime, endolymph, perilymph, feces, gastric acid, gastric juice, lymph,mucus (including nasal drainage and phlegm), pericardial fluid,peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum (skin oil),semen, sputum, sweat, synovial fluid, tears, vomit, urine and exhaledcondensate.

In certain embodiments, a subject biosensor can be used diagnose apathogen infection by detecting a target nucleic acid from a pathogen ina sample. The target nucleic acid can be, for example, from a virus thatis selected from the group comprising human immunodeficiency virus 1 and2 (HIV-1 and HIV-2), human T-cell leukaemia virus and 2 (HTLV-1 andHTLV-2), respiratory syncytial virus (RSV), adenovirus, hepatitis Bvirus (HBV), hepatitis C virus (HCV), Epstein-Barr virus (EBV), humanpapillomavirus (HPV), varicella zoster virus (VZV), cytomegalovirus(CMV), herpes-simplex virus 1 and 2 (HSV-1 and HSV-2), human herpesvirus8 (HHV-8, also known as Kaposi sarcoma herpesvirus) and flaviviruses,including yellow fever virus, dengue virus, Japanese encephalitis virus,West Nile virus and Ebola virus. The present invention is not, however,limited to the detection of nucleic acid, e.g., DNA or RNA, sequencesfrom the aforementioned viruses, but can be applied without any problemto other pathogens important in veterinary and/or human medicine.

Human papillomaviruses (HPV) are further subdivided on the basis oftheir DNA sequence homology into more than 70 different types. Thesetypes cause different diseases. HPV types 1, 2, 3, 4, 7, 10 and 26-29cause benign warts. HPV types 5, 8, 9, 12, 14, 15, 17 and 19-25 and46-50 cause lesions in patients with a weakened immune system. Types 6,11, 34, 39, 41-44 and 51-55 cause benign acuminate warts on the mucosaeof the genital region and of the respiratory tract. HPV types 16 and 18are of special medical interest, as they cause epithelial dysplasias ofthe genital mucosa and are associated with a high proportion of theinvasive carcinomas of the cervix, vagina, vulva and anal canal.Integration of the DNA of the human papillomavirus is considered to bedecisive in the carcinogenesis of cervical cancer. Humanpapillomaviruses can be detected for example from the DNA sequence oftheir capsid proteins L1 and L2. Accordingly, the method of the presentinvention is especially suitable for the detection of DNA sequences ofHPV types 16 and/or 18 in tissue samples, for assessing the risk ofdevelopment of carcinoma.

In some cases, the nanosensor can be employed to detect a biomarker thatis present at a low concentration. For example, the nanosensor can beused to detect cancer antigens in a readily accessible bodily fluids(e.g., blood, saliva, urine, tears, etc.), to detect biomarkers fortissue-specific diseases in a readily accessible bodily fluid (e.g., abiomarkers for a neurological disorder (e.g., Alzheimer's antigens)), todetect infections (particularly detection of low titer latent viruses,e.g., HIV), to detect fetal antigens in maternal blood, and fordetection of exogenous compounds (e.g., drugs or pollutants) in asubject's bloodstream, for example.

The following table provides a list of protein biomarkers that can bedetected using the subject nanosensor (when used in conjunction with anappropriate monoclonal antibody), and their associated diseases. Onepotential source of the biomarker (e.g., “CSF”; cerebrospinal fluid) isalso indicated in the table. In many cases, the subject biosensor candetect those biomarkers in a different bodily fluid to that indicated.For example, biomarkers that are found in CSF can be identified inurine, blood or saliva.

H) Utility

The subject method finds use in a variety of different applicationswhere determination of the presence or absence, and/or quantification ofone or more analytes in a sample are desired. For example, the subjectmethod finds use in the detection of proteins, peptides, nucleic acids,synthetic compounds, inorganic compounds, and the like.

In certain embodiments, the subject method finds use in the detection ofnucleic acids, proteins, or other biomolecules in a sample. The methodscan include the detection of a set of biomarkers, e.g., two or moredistinct protein or nucleic acid biomarkers, in a sample. For example,the methods can be used in the rapid, clinical detection of two or moredisease biomarkers in a biological sample, e.g., as can be employed inthe diagnosis of a disease condition in a subject, or in the ongoingmanagement or treatment of a disease condition in a subject, etc. Asdescribed above, communication to a physician or other health-careprovider can better ensure that the physician or other health-careprovider is made aware of, and cognizant of, possible concerns and canthus be more likely to take appropriate action.

The applications of the devices, systems and methods in the presentinvention of employing a CROF device include, but are not limited to,(a) the detection, purification and quantification of chemical compoundsor biomolecules that correlates with the stage of certain diseases,e.g., infectious and parasitic disease, injuries, cardiovasculardisease, cancer, mental disorders, neuropsychiatric disorders andorganic diseases, e.g., pulmonary diseases, renal diseases, (b) thedetection, purification and quantification of microorganism, e.g.,virus, fungus and bacteria from environment, e.g., water, soil, orbiological samples, e.g., tissues, bodily fluids, (c) the detection,quantification of chemical compounds or biological samples that posehazard to food safety or national security, e.g. toxic waste, anthrax,(d) quantification of vital parameters in medical or physiologicalmonitor, e.g., glucose, blood oxygen level, total blood count, (e) thedetection and quantification of specific DNA or RNA from biosamples,e.g., cells, viruses, bodily fluids, (f) the sequencing and comparing ofgenetic sequences in DNA in the chromosomes and mitochondria for genomeanalysis or (g) to detect reaction products, e.g., during synthesis orpurification of pharmaceuticals. Some of the specific applications ofthe devices, systems and methods in the present invention are describednow in further detail.

I) Diagnostic Method

In certain embodiments, the subject method finds use in detectingbiomarkers. In certain embodiments, the devices, systems and methods inthe present invention of using CROF are used to detect the presence orabsence of particular biomarkers, as well as an increase or decrease inthe concentration of particular biomarkers in blood, plasma, serum, orother bodily fluids or excretions, such as but not limited to urine,blood, serum, plasma, saliva, semen, prostatic fluid, nipple aspiratefluid, lachrymal fluid, perspiration, feces, cheek swabs, cerebrospinalfluid, cell lysate samples, amniotic fluid, gastrointestinal fluid,biopsy tissue, and the like. Thus, the sample, e.g. a diagnostic sample,can include various fluid or solid samples.

In some instances, the sample can be a bodily fluid sample from asubject who is to be diagnosed. In some instances, solid or semi-solidsamples can be provided. The sample can include tissues and/or cellscollected from the subject. The sample can be a biological sample.Examples of biological samples can include but are not limited to,blood, serum, plasma, a nasal swab, a nasopharyngeal wash, saliva,urine, gastric fluid, spinal fluid, tears, stool, mucus, sweat, earwax,oil, a glandular secretion, cerebral spinal fluid, tissue, semen,vaginal fluid, interstitial fluids derived from tumorous tissue, ocularfluids, spinal fluid, a throat swab, breath, hair, finger nails, skin,biopsy, placental fluid, amniotic fluid, cord blood, lymphatic fluids,cavity fluids, sputum, pus, microbiota, meconium, breast milk, exhaledcondensate and/or other excretions. The samples can includenasopharyngeal wash. Nasal swabs, throat swabs, stool samples, hair,finger nail, ear wax, breath, and other solid, semi-solid, or gaseoussamples can be processed in an extraction buffer, e.g., for a fixed orvariable amount of time, prior to their analysis. The extraction bufferor an aliquot thereof can then be processed similarly to other fluidsamples if desired. Examples of tissue samples of the subject caninclude but are not limited to, connective tissue, muscle tissue,nervous tissue, epithelial tissue, cartilage, cancerous sample, or bone.

In some instances, the subject from which a diagnostic sample isobtained can be a healthy individual, or can be an individual at leastsuspected of having a disease or a health condition. In some instances,the subject can be a patient.

In certain embodiments, the CROF device includes a capture agentconfigured to specifically bind a biomarker in a sample provided by thesubject. In certain embodiments, the biomarker can be a protein. Incertain embodiments, the biomarker protein is specifically bound by anantibody capture agent present in the CROF device. In certainembodiments, the biomarker is an antibody specifically bound by anantigen capture agent present in the CROF device. In certainembodiments, the biomarker is a nucleic acid specifically bound by anucleic acid capture agent that is complementary to one or both strandsof a double-stranded nucleic acid biomarker, or complementary to asingle-stranded biomarker. In certain embodiments, the biomarker is anucleic acid specifically bound by a nucleic acid binding protein. Incertain embodiments, the biomarker is specifically bound by an aptamer.

The presence or absence of a biomarker or significant changes in theconcentration of a biomarker can be used to diagnose disease risk,presence of disease in an individual, or to tailor treatments for thedisease in an individual. For example, the presence of a particularbiomarker or panel of biomarkers can influence the choices of drugtreatment or administration regimes given to an individual. Inevaluating potential drug therapies, a biomarker can be used as asurrogate for a natural endpoint such as survival or irreversiblemorbidity. If a treatment alters the biomarker, which has a directconnection to improved health, the biomarker can serve as a surrogateendpoint for evaluating the clinical benefit of a particular treatmentor administration regime. Thus, personalized diagnosis and treatmentbased on the particular biomarkers or panel of biomarkers detected in anindividual are facilitated by the subject method. Furthermore, the earlydetection of biomarkers associated with diseases is facilitated by thehigh sensitivity of the devices, systems and methods in the presentinvention, as described above. Due to the capability of detectingmultiple biomarkers with a mobile device, such as a smartphone, combinedwith sensitivity, scalability, and ease of use, the presently disclosedmethod finds use in portable and point-of-care or near-patient moleculardiagnostics.

In certain embodiments, the subject method finds use in detectingbiomarkers for a disease or disease state. In certain instances, thesubject method finds use in detecting biomarkers for thecharacterization of cell signaling pathways and intracellularcommunication for drug discovery and vaccine development. For example,the subject method can be used to detect and/or quantify the amount ofbiomarkers in diseased, healthy or benign samples. In certainembodiments, the subject method finds use in detecting biomarkers for aninfectious disease or disease state. In some cases, the biomarkers canbe molecular biomarkers, such as but not limited to proteins, nucleicacids, carbohydrates, small molecules, and the like.

The subject method find use in diagnostic assays, such as, but notlimited to, the following: detecting and/or quantifying biomarkers, asdescribed above; screening assays, where samples are tested at regularintervals for asymptomatic subjects; prognostic assays, where thepresence and or quantity of a biomarker is used to predict a likelydisease course; stratification assays, where a subject's response todifferent drug treatments can be predicted; efficacy assays, where theefficacy of a drug treatment is monitored; and the like.

In certain embodiments, a subject biosensor can be used diagnose apathogen infection by detecting a target nucleic acid from a pathogen ina sample. The target nucleic acid can be, for example, from a virus thatis selected from the group comprising human immunodeficiency virus 1 and2 (HIV-1 and HIV-2), human T-cell leukaemia virus and 2 (HTLV-1 andHTLV-2), respiratory syncytial virus (RSV), adenovirus, hepatitis Bvirus (HBV), hepatitis C virus (HCV), Epstein-Barr virus (EBV), humanpapillomavirus (HPV), varicella zoster virus (VZV), cytomegalovirus(CMV), herpes-simplex virus 1 and 2 (HSV-1 and HSV-2), human herpesvirus8 (HHV-8, also known as Kaposi sarcoma herpesvirus) and flaviviruses,including yellow fever virus, dengue virus, Japanese encephalitis virus,West Nile virus and Ebola virus. The present invention is not, however,limited to the detection of nucleic acid, e.g., DNA or RNA, sequencesfrom the aforementioned viruses, but can be applied without any problemto other pathogens important in veterinary and/or human medicine.

Human papillomaviruses (HPV) are further subdivided on the basis oftheir DNA sequence homology into more than 70 different types. Thesetypes cause different diseases. HPV types 1, 2, 3, 4, 7, 10 and 26-29cause benign warts. HPV types 5, 8, 9, 12, 14, 15, 17 and 19-25 and46-50 cause lesions in patients with a weakened immune system. Types 6,11, 34, 39, 41-44 and 51-55 cause benign acuminate warts on the mucosaeof the genital region and of the respiratory tract. HPV types 16 and 18are of special medical interest, as they cause epithelial dysplasias ofthe genital mucosa and are associated with a high proportion of theinvasive carcinomas of the cervix, vagina, vulva and anal canal.Integration of the DNA of the human papillomavirus is considered to bedecisive in the carcinogenesis of cervical cancer. Humanpapillomaviruses can be detected for example from the DNA sequence oftheir capsid proteins L1 and L2. Accordingly, the method of the presentinvention is especially suitable for the detection of DNA sequences ofHPV types 16 and/or 18 in tissue samples, for assessing the risk ofdevelopment of carcinoma.

Other pathogens that can be detected in a diagnostic sample using thedevices, systems and methods in the present invention include, but arenot limited to: Varicella zoster; Staphylococcus epidermidis,Escherichia coli, methicillin-resistant Staphylococcus aureus (MSRA),Staphylococcus aureus, Staphylococcus hominis, Enterococcus faecalis,Pseudomonas aeruginosa, Staphylococcus capitis, Staphylococcus warneri,Klebsiella pneumoniae, Haemophilus influenzae, Staphylococcus simulans,Streptococcus pneumoniae and Candida albicans; gonorrhea (Neisseriagorrhoeae), syphilis (Treponena pallidum), clamydia (Clamydatracomitis), nongonococcal urethritis (Ureaplasm urealyticum), chancroid(Haemophilus ducreyi), trichomoniasis (Trichomonas vaginalis);Pseudomonas aeruginosa, methicillin-resistant Staphylococcus aureus(MSRA), Klebsiella pneumoniae, Haemophilis influenzae, Staphylococcusaureus, Stenotrophomonas maltophilia, Haemophilis parainfluenzae,Escherichia coli, Enterococcus faecalis, Serratia marcescens,Haemophilis parahaemolyticus, Enterococcus cloacae, Candida albicans,Moraxiella catarrhalis, Streptococcus pneumoniae, Citrobacter freundii,Enterococcus faecium, Klebsiella oxytoca, Pseudomonas fluorescens,Neisseria meningitidis, Streptococcus pyogenes, Pneumocystis carinii,Klebsiella pneumoniae Legionella pneumophila, Mycoplasma pneumoniae, andMycobacterium tuberculosis, etc.

In some cases, the CROF device can be employed to detect a biomarkerthat is present at a low concentration. For example, the CROF device canbe used to detect cancer antigens in a readily accessible bodily fluids(e.g., blood, saliva, urine, tears, etc.), to detect biomarkers fortissue-specific diseases in a readily accessible bodily fluid (e.g., abiomarkers for a neurological disorder (e.g., Alzheimer's antigens)), todetect infections (particularly detection of low titer latent viruses,e.g., HIV), to detect fetal antigens in maternal blood, and fordetection of exogenous compounds (e.g., drugs or pollutants) in asubject's bloodstream, for example.

One potential source of the biomarker (e.g., “CSF”; cerebrospinal fluid)is also indicated in the table. In many cases, the subject biosensor candetect those biomarkers in a different bodily fluid to that indicated.For example, biomarkers that are found in CSF can be identified inurine, blood or saliva. It will also be clear to one with ordinary skillin the art that the subject CROF devices can be configured to captureand detect many more biomarkers known in the art that are diagnostic ofa disease or health condition.

A biomarker can be a protein or a nucleic acid (e.g., mRNA) biomarker,unless specified otherwise. The diagnosis can be associated with anincrease or a decrease in the level of a biomarker in the sample, unlessspecified otherwise. Lists of biomarkers, the diseases that they can beused to diagnose, and the sample in which the biomarkers can be detectedare described in Tables 1 and 2 of U.S. provisional application Ser. No.62/234,538, filed on Sep. 29, 2015, which application is incorporated byreference herein.

In some instances, the devices, systems and methods in the presentinvention is used to inform the subject from whom the sample is derivedabout a health condition thereof. Health conditions that can bediagnosed or measured by the devices, systems and methods in the presentinvention, device and system include, but are not limited to: chemicalbalance; nutritional health; exercise; fatigue; sleep; stress;prediabetes; allergies; aging; exposure to environmental toxins,pesticides, herbicides, synthetic hormone analogs; pregnancy; menopause;and andropause. Table 3 of U.S. provisional application Ser. No.62/234,538, filed on Sep. 29, 2015, which application is incorporated byreference herein, provides a list of biomarker that can be detectedusing the present CROF device (when used in conjunction with anappropriate monoclonal antibody, nucleic acid, or other capture agent),and their associated health conditions.

J) Kits

Aspects of the present disclosure include a kit that find use inperforming the devices, systems and methods in the present invention, asdescribed above. In certain embodiments, the kit includes instructionsfor practicing the subject methods using a hand held device, e.g., amobile phone. These instructions can be present in the subject kits in avariety of forms, one or more of which can be present in the kit. Oneform in which these instructions can be present is as printedinformation on a suitable medium or substrate, e.g., a piece or piecesof paper on which the information is printed, in the packaging of thekit, in a package insert, etc. Another means would be a computerreadable medium, e.g., diskette, CD, DVD, Blu-Ray, computer-readablememory, etc., on which the information has been recorded or stored. Yetanother means that can be present is a website address which can be usedvia the Internet to access the information at a removed site. The kitcan further include a software for implementing a method for measuringan analyte on a device, as described herein, provided on a computerreadable medium. Any convenient means can be present in the kits.

In certain embodiments, the kit includes a detection agent that includesa detectable label, e.g. a fluorescently labeled antibody oroligonucleotide that binds specifically to an analyte of interest, foruse in labeling the analyte of interest. The detection agent can beprovided in a separate container as the CROF device, or can be providedin the CROF device.

In certain embodiments, the kit includes a control sample that includesa known detectable amount of an analyte that is to be detected in thesample. The control sample can be provided in a container, and can be insolution at a known concentration, or can be provided in dry form, e.g.,lyophilized or freeze dried. The kit can also include buffers for use indissolving the control sample, if it is provided in dry form.

FIG. 9 depicts a block diagram of a computer system operating inaccordance with one or more aspects of the present disclosure. Invarious illustrative examples, computer system 600 may be system 2 ofFIG. 4 .

In certain implementations, computer system 600 may be connected (e.g.,via a network, such as a Local Area Network (LAN), an intranet, anextranet, or the Internet) to other computer systems. Computer system600 may operate in the capacity of a server or a client computer in aclient-server environment, or as a peer computer in a peer-to-peer ordistributed network environment. Computer system 600 may be provided bya personal computer (PC), a tablet PC, a set-top box (STB), a PersonalDigital Assistant (PDA), a cellular telephone, a web appliance, aserver, a network router, switch or bridge, or any device capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that device. Further, the term “computer” shallinclude any collection of computers that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methods described herein.

In a further aspect, the computer system 600 may include a processingdevice 602, a volatile memory 604 (e.g., random access memory (RAM)), anon-volatile memory 606 (e.g., read-only memory (ROM) orelectrically-erasable programmable ROM (EEPROM)), and a data storagedevice 616, which may communicate with each other via a bus 608.

Processing device 602 may be provided by one or more processors such asa general purpose processor (such as, for example, a complex instructionset computing (CISC) microprocessor, a reduced instruction set computing(RISC) microprocessor, a very long instruction word (VLIW)microprocessor, a microprocessor implementing other types of instructionsets, or a microprocessor implementing a combination of types ofinstruction sets) or a specialized processor (such as, for example, anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), or a networkprocessor).

Computer system 600 may further include a network interface device 622.Computer system 600 also may include a video display unit 610 (e.g., anLCD), an alphanumeric input device 612 (e.g., a keyboard), a cursorcontrol device 614 (e.g., a mouse), and a signal generation device 620.

Data storage device 616 may include a non-transitory computer-readablestorage medium 624 on which may store instructions 626 encoding any oneor more of the methods or functions described herein, includinginstructions of the training and applying machine learning model 108 asshown in FIG. 4 .

Instructions 626 may also reside, completely or partially, withinvolatile memory 604 and/or within processing device 602 during executionthereof by computer system 600, hence, volatile memory 604 andprocessing device 602 may also constitute machine-readable storagemedia.

While computer-readable storage medium 624 is shown in the illustrativeexamples as a single medium, the term “computer-readable storage medium”shall include a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more sets of executable instructions. The term“computer-readable storage medium” shall also include any tangiblemedium that is capable of storing or encoding a set of instructions forexecution by a computer that cause the computer to perform any one ormore of the methods described herein. The term “computer-readablestorage medium” shall include, but not be limited to, solid-statememories, optical media, and magnetic media.

The methods, components, and features described herein may beimplemented by discrete hardware components or may be integrated in thefunctionality of other hardware components such as ASICS, FPGAs, DSPs orsimilar devices. In addition, the methods, components, and features maybe implemented by firmware modules or functional circuitry withinhardware devices. Further, the methods, components, and features may beimplemented in any combination of hardware devices and computer programcomponents, or in computer programs.

FIG. 10 (A) illustrates that the monitoring marks in the presentinvention are actually adding known meta structures into a lattice. Whena lattice gets distorted, one always can use the knowledge of the metastructure to recovering to nearly perfect structure, hence greatlyimprove the accuracy of the machine learning model and the assayaccuracy.

the mapping between a crystalline structure and an amorphous structure.(B) illustrates the mapping between a crystalline structure injectedwith a meta structure and an amorphous structure injected with the metastructure according to an embodiment of the disclosure.

FIG. 11 illustrate an actual examples of the difference between trainingmachine learning without using the monitoring marks (FIG. 10 (a0) whichneed many samples and long time to train and the test generatesartifacts (missing cells and create the non-exist cells), while usingthe monitoring marks, the training sample number and time aresignificantly reduced, and the test do not generate the artifacts.

Unless specifically stated otherwise, terms such as “receiving,”“associating,” “determining,” “updating” or the like, refer to actionsand processes performed or implemented by computer systems thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system registers and memories into otherdata similarly represented as physical quantities within the computersystem memories or registers or other such information storage,transmission or display devices. Also, the terms “first,” “second,”“third,” “fourth,” etc. as used herein are meant as labels todistinguish among different elements and may not have an ordinal meaningaccording to their numerical designation.

Examples described herein also relate to an apparatus for performing themethods described herein. This apparatus may be specially constructedfor performing the methods described herein, or it may comprise ageneral purpose computer system selectively programmed by a computerprogram stored in the computer system. Such a computer program may bestored in a computer-readable tangible storage medium.

The methods and illustrative examples described herein are notinherently related to any particular computer or other apparatus.Various general purpose systems may be used in accordance with theteachings described herein, or it may prove convenient to construct morespecialized apparatus to perform method 300 and/or each of itsindividual functions, routines, subroutines, or operations. Examples ofthe structure for a variety of these systems are set forth in thedescription above.

The above description is intended to be illustrative, and notrestrictive. Although the present disclosure has been described withreferences to specific illustrative examples and implementations, itwill be recognized that the present disclosure is not limited to theexamples and implementations described. The scope of the disclosureshould be determined with reference to the following claims, along withthe full scope of equivalents to which the claims are entitled.

What is claimed is:
 1. A method of training a machine learning model foran image based assay, wherein a sample in the assay is, during a test,imaged by an imaging system with an imperfection, and wherein the samplecontains or is suspected of containing an analyte; comprising: havingthe sample forming a thin layer on an imaging area of a sample holder,wherein the sample holder comprises one or more monitoring marks in theimaging area, and wherein at least one of the geometric and/or opticalproperties of the one or more monitoring marks is predetermined andknown; imaging, using the imaging system, an original image of thesample on the imaging area of the sample holder; correcting animperfection in the original image using the at least one of thegeometric and/or optical properties of the one or more monitoring marks,to generate a corrected image; and training a machine learning modelusing the corrected image to generate a trained model for measuring theanalyte.
 2. The method of claim 1, wherein correcting the imperfectionin the original image to generate the corrected image comprises applyinga spatial transform to the original image, wherein the spatialtransformation uses a mapping between the positions of the one or moremonitoring marks in the original image and the predetermined and knownpositions of the one or more monitoring marks in the sample holder.
 3. Amethod of an image based assay using a machine learning model, wherein asample in the assay is, during a test, imaged by an imaging system withan imperfection, and wherein the sample contains or is suspected ofcontaining an analyte, the method comprising: receiving an originalimage, imaged by the imaging system, of an imaging area of the sampleholder, wherein the imaging area of the sample holder comprises a sampleand one or more monitoring marks; wherein at least one of the geometricand/or optical properties of the one or more monitoring marks ispredetermined and known; correcting an imperfection in the originalimage using the at least one of the geometric and/or optical propertiesof the one or more monitoring marks to generate a corrected image; andanalyzing the analyte using the machine learning model.
 4. The method ofclaim 3, wherein correcting the imperfection in the original image togenerate the corrected image comprises applying a spatial transform tothe original image, wherein the spatial transformation uses a mappingbetween the positions of the one or more monitoring marks in theoriginal image and the predetermined and known positions of the one ormore monitoring marks in the sample holder.
 5. The method of claim 3,wherein the machine learning model is trained by the method of claim 1.6. The method of claim 4, wherein the machine learning model is trainedby the method of claim
 2. 7. An image-based assay system, wherein asample in the assay is, during a test, imaged by an imaging system withan imperfection, and wherein the sample contains or is suspected ofcontaining an analyte, the system comprising: a database system to storeimages; and a processing device, communicatively coupled to the databasesystem, to: receive an original image, captured by the imaging system,of an imaging area of the sample holder, wherein the imaging area of thesample holder comprises a sample and one or more monitoring marks;wherein at least one of the geometric and/or optical properties of themonitoring marks is predetermined and known; correct an imperfection inthe original image using the at least one of the geometric and/oroptical properties of the one or more monitoring marks, generating acorrected image; and analyze the corrected image using a machinelearning model.
 8. The system of claim 7, wherein the machine learningmodel is trained by the method of claim 1, wherein the training uses animage that is corrected using the one or more monitoring marks on asecond sample holder which is identical to the sample holder in claim 7.9. The system of claim 7, wherein correcting the imperfection in theoriginal image to generate the corrected image comprises a spatialtransform that uses a mapping between the positions of the one or moremonitoring marks in the original image and the predetermined and knownpositions of one or more monitoring marks in the sample holder.
 10. Themethod of claim 2, wherein the analyte is a cell.
 11. The method ofclaim 3, wherein the analyte is a cell.
 12. The method of claim 1,wherein the sample holder comprises a first plate, a second plate, andthe one or more monitoring marks, and wherein the one or more monitoringmarks comprise pillars at the predetermined positions on at least one ofthe first plate and the second plate, wherein the first and secondplates have a spacing of 200 um or less, and wherein the sample issandwiched between the first and second plates.
 13. The method of claim3, wherein the sample holder comprises a first plate, a second plate,and the one or more monitoring marks, and wherein the one or moremonitoring marks comprise pillars at the predetermined positions on atleast one of the first plate and the second plate, wherein the first andsecond plates have a spacing of 200 um or less, and wherein the sampleis sandwiched between the first and second plates.
 14. The system ofclaim 7, wherein the sample holder comprises a first plate, a secondplate, and the one or more monitoring marks, and wherein the one or moremonitoring marks comprise pillars at the predetermined positions on atleast one of the first plate and the second plate, wherein the first andsecond plates have a spacing of 200 um or less, and wherein the sampleis sandwiched between the first and second plates.
 15. The method ofclaim 1, wherein the predetermined positions of the one or moremonitoring marks are positioned periodically on the sample holder with aperiodicity value of 200 um or less.
 16. The method of claim 3, whereinthe predetermined positions of the one or more monitoring marks arepositioned periodically on the sample holder with a periodicity value of200 um or less.
 17. The system of claim 7, wherein the predeterminedpositions of the one or more monitoring marks are positionedperiodically on the sample holder with a periodicity value of 200 um orless.
 18. The method of claim 1, wherein the predetermined positions ofthe one or more monitoring marks in the first image are distributedperiodically with at least one periodicity value, and wherein detectingthe locations of the one or more monitoring marks in the first imagecomprises: detecting, using a second machine learning model, thelocations of the one or more monitoring marks in the first image; andcorrecting, using the at least one periodicity value, an error in thedetected locations of the one or more monitoring marks in the firstimage.
 19. The method of claim 3, wherein the predetermined positions ofthe one or more monitoring marks in the first image are distributedperiodically with at least one periodicity value, and wherein detectingthe locations of the one or more monitoring marks in the first imagecomprises: detecting, using a second machine learning model, thelocations of the one or more monitoring marks in the first image; andcorrecting, using the at least one periodicity value, an error in thedetected locations of the one or more monitoring marks in the firstimage.
 20. The system of claim 7, wherein: the predetermined positionsof the one or more monitoring marks in the first image are distributedperiodically with at least one periodicity value, and the processingdevice detects the locations of the one or more monitoring marks in thefirst image, comprising: (i) detecting, using a second machine learningmodel, the locations of the one or more monitoring marks in the firstimage; and (ii) correcting, using the at least one periodicity value, anerror in the detected locations of the one or more monitoring marks inthe first image.
 21. The method of claim 1, wherein the sample holdercomprises a first plate and a second plate, wherein the monitoring markscomprise pillars at the predetermined positions on at least one of thefirst plate and the second plate, and wherein the sample is sandwichedbetween the first and second plates.
 22. The method of claim 3, whereinthe sample holder comprises a first plate and a second plate, whereinthe monitoring marks comprise pillars at the predetermined positions onat least one of the first plate and the second plate, and wherein thesample is sandwiched between the first and second plates.
 23. The systemof claim 7, wherein the sample holder comprises a first plate and asecond plate, wherein the monitoring marks comprise pillars at thepredetermined positions on at least one of the first plate and thesecond plate, and wherein the sample is sandwiched between the first andsecond plates.
 24. The method of claim 1, wherein: the sample holdercomprises a first plate and a second plate that are movable to eachother in different configurations, including an open configuration and aclosed configuration, and the sample is between the first and secondplates; a plurality of spacers has a height of 200 um or less, and is onone of the two plates or both, and the plurality of spacers are situatedbetween the two plates at the closed configuration; and the samplethickness is regulated by the spacers and the first and second plate;wherein the open configuration is the configuration, in which, the twoplates are separated apart, the spacing between the plates is notregulated by the spacers, and the sample is deposited on one or both ofthe plates; and wherein the closed configuration, which is configuredafter the sample deposition in the open configuration, is theconfiguration, in which, at least part of the sample is compressed bythe two plates into a layer of substantially uniform thickness, whereinthe substantially uniform thickness of the layer is regulated by the twoplates and the spacers.
 25. The system of claim 7, wherein: the sampleholder comprises a first plate and a second plate that are movable toeach other in different configurations, including an open configurationand a closed configuration, and the sample is between the first andsecond plates; a plurality of spacers has a height of 200 um or less,and is on one of the two plates or both, and the plurality of spacersare situated between the two plates at the closed configuration; and thesample thickness is regulated by the spacers and the first and secondplate; wherein the open configuration is the configuration, in which,the two plates are separated apart, the spacing between the plates isnot regulated by the spacers, and the sample is deposited on one or bothof the plates; and wherein the closed configuration, which is configuredafter the sample deposition in the open configuration, is theconfiguration, in which, at least part of the sample is compressed bythe two plates into a layer of substantially uniform thickness, whereinthe substantially uniform thickness of the layer is regulated by the twoplates and the spacers.
 26. The method of claim 18, wherein the spacersare the monitoring markers.
 27. The system of claim 19, wherein thespacers are the monitoring markers.
 28. The method of claim 1, whereinthe one or more monitoring marks comprise at least four monitoring marksof a pillar shape, wherein the four monitoring marks define four cornersin a defined region in the image region of the sample holder, and themethod is further comprising: detecting the locations of the fourmonitoring marks in the image; performing a spatial transform of thedefined region using a mapping between the locations of the fourmonitoring marks the predetermined positions of the four monitoringmarks in the sample holder, to generate the corrected image.
 29. Themethod of claim 3, wherein the one or more monitoring marks comprise atleast four monitoring marks of a pillar shape, wherein the fourmonitoring marks define four corners in a defined region in the imageregion of the sample holder, and the method is further comprising:detecting the locations of the four monitoring marks in the image;performing a spatial transform of the defined region using a mappingbetween the locations of the four monitoring marks the predeterminedpositions of the four monitoring marks in the sample holder, to generatethe corrected image.
 30. The system of claim 7, wherein the one or moremonitoring marks comprise at least four monitoring marks of a pillarshape, wherein the four monitoring marks define four corners in adefined region in the image region of the sample holder, and the systemis further to: detect the locations of the four monitoring marks in theimage; perform a spatial transform of the defined region using a mappingbetween the locations of the four monitoring marks the predeterminedpositions of the four monitoring marks in the sample holder, to generatethe corrected image.
 31. The system of claim 7, wherein the thickness ofthe sample layer is 100 um or less.
 32. The method of claim 1, whereinthe monitoring markers have a pillar shape and arranged periodically.33. The method of claim 2, wherein the monitoring markers have a pillarshape and are arranged periodically.
 34. The method of claim 3, whereinthe monitoring markers have a pillar shape and are arrangedperiodically.
 35. The system of claim 7, wherein the monitoring markershave a pillar shape and arranged periodically.
 36. The method of claim1, wherein the sample is selected from cells, tissues, bodily fluids,stool, amniotic fluid, aqueous humour, vitreous humour, blood, wholeblood, fractionated blood, plasma, serum, breast milk, cerebrospinalfluid, cerumen, chyle, chime, endolymph, perilymph, feces, gastric acid,gastric juice, lymph, mucus, nasal drainage, phlegm, pericardial fluid,peritoneal fluid, pleural fluid, pus, rheum, saliva, sebum, semen,sputum, sweat, synovial fluid, tears, vomit, urine, and exhaled breathcondensate.
 37. The method of claim 1, wherein the analyte comprisesproteins, nucleic acids, and other biomolecules.
 38. The method of claim1, wherein the image based assay is for blood cell counting, and theanalyte comprises cells.
 39. The method of claim 3, wherein animperfection in the image to be corrected comprises an imperfection inoptical components and conditions including but not limit to opticalattenuator, beam splitter, depolarizer, diaphragm, diffractive beamsplitter, diffuser, ground glass, lens, littrow prism, multifocaldiffractive lens, nanophotonic resonator, nuller, optical circulator,optical isolator, optical microcavity, photonic integrated circuit,pinhole (optics), polarizer, primary mirror, prism, q-plate,retroreflector, spatial filter, spatial light modulator, virtuallyimaged phased array, waveguide (optics), waveplate, or zone plate. 40.The method of claim 3, wherein an imperfection in the image to becorrected comprises an imperfection in light illumination and conditionsincluding but not limit to light source intensity, light source spectra,light source color, light source direction, light source brightness,light source contrast, light source wavelength band width, light sourcecoherence, light source phase, light source polarization.